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contract  with  Army  Material  Command  was  done  to  determine  what  improvements  could  be  made  to  the 
debriefing  process.  The  respondents  indicated  that  many  modifications  could  be  done  to  improve  the 
process. 
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I .   INTRODUCTION 

A.   BACKGROUND 

Competition  requirements  for  defense  procurement  under 
conditions  that  do  not  need  a  waiver  require  two  or  more 
defense  contractors  to  compete  for  contract  award  on  a 
solicitation  [_4:  p.  2]  .  The  defense  agency  selects  a  winner  or 
winners  of  the  competition  and  the  other  contractors  are 
unsuccessful  offerors  for  that  particular  competition.  When 
price  alone  is  not  the  only  criteria  used  to  determine 
contract  award  the  unsuccessful  offerors  may  be  very 
interested  to  understand  why  they  did  not  win  the  contract 
award.  By  requesting  a  debriefing  from  the  agency  that  is 
responsible  for  the  procurement,  unsuccessful  offerors  can 
obtain  the  information  they  need  to  determine  why  they  lost 
the  procurement  competition.  The  debriefing  is  a  meeting 
between  representatives  of  the  procuring  agency  and 
unsuccessful  company  where  agency  representatives  will  explain 
to  the  company  why  they  did  not  win  the  contract  award. 

Unsuccessful  offerors  do  not  receive  formal  debriefings 
for  all  defense  procurement  actions.  Rather  only  one  of  the 
two  methods  for  federal  procurement  warrants  a  formal 
debriefing  [7_:  p. 15-41].  The  two  possible  methods  of 
procuring  goods  and  services  in  federal  procurement  are  sealed 


bidding  and  negotiated  procurement.  The  first  method,  sealed 
bidding  is  to  be  used  when  (a)  there  is  adequate  time,  (b) 
contract  award  will  be  made  based  on  price  and  price  related 
factors,  (c)  no  discussions  with  offerors  are  needed,  and  (d) 
there  is  an  expectation  of  receiving  bids  from  more  than  one 
offeror  [4:  p.l]  .  The  second  method,  competitive  negotiation, 
is  to  be  used  if  the  procurement  doesn't  meet  the  criteria  of 
sealed  bidding. 

The  reason  for  contract  award  using  sealed  bidding  is  easy 
to  determine.  The  lowest  priced  bid  from  the  offeror  who  is 
both  responsible  and  responsive  wins  the  contract  award 
[9:  p. 3-23].  Thus,  unsuccessful  offerors  generally  know 
exactly  why  they  didn't  win  the  contract  award  and  it  is 
normally  because  they  didn't  have  the  lowest  priced  bid. 

For  competitive  negotiations  the  reason  for  not  winning 
contract  award  may  not  be  so  easy  to  determine.  With  the 
advent  of  best  value  contracting,  discovering  the  reasons  for 
not  winning  contract  award  may  be  further  clouded  from  view. 
This  is  because  in  this  type  of  contracting,  contract  award  is 
made  based  on  cost  plus  one  or  more  other  factors.  Examples 
of  other  factors  could  be  technical  solution,  management 
experience,  past  performance,  and  risk.  In  addition,  the 
factors  are  weighted  in  accordance  with  their  importance  to 
the  contract  award.  Once  an  evaluation  is  made  and  a  winner 
is  selected,  the  unsuccessful  offerors  are  notified  that  they 
were  not  selected  for  contract  award.   With  a  multitude  of 


selection  criteria  and  different  weights  used  for  each 
criteria,  unsuccessful  offerors  may  have  little  idea  why  they 
lost  the  contract  award.  It  is  no  longer  a  case  of  realizing 
that  his  cost  was  too  high.  It  may  be  that  his  technical 
solution,  or  management  experience  or  some  other  factor  or 
combination  of  factors,  when  evaluated  in  accordance  with  the 
evaluation  criteria,  were  not  as  good  as  the  winner's.  The 
unsuccessful  offeror  can  request  a  debriefing  from  the  agency 
that  is  responsible  for  the  procurement  to  find  out  what  the 
actual  reasons  are  for  him  not  being  selected  for  contract 
award . 

With  the  recent  cuts  in  military  procurement  resulting 
from  the  end  of  the  cold  war  debriefings  have  increased  in 
importance  for  defense  contractors  as  they  fight  for  a  piece 
of  the  remaining  procurement  pie  [12] .  By  understanding  why 
they  lost  one  award,  defense  contractors  can  use  the 
information  they  obtained  from  a  debriefing  to  increase  their 
competitive  position  for  the  next  procurement. 

Debriefings  also  have  the  potential  to  help  the  Department 
of  Defense  (DOD) .  From  it's  peak  in  fiscal  year  (FY)  1985  the 
defense  procurement  budget  will  decrease  approximately  65%  to 
a  projected  $43.3  billion  for  FY  1995  [8:  p. 8].  Budget 
constraints  caused  the  Army  alone  to  cut  57  procurement 
programs  and  scale  back  on  77  others  in  FY  1993  [16:  p. 10]. 
The  dollars  left  for  procurement  must  be  wisely  and 
efficiently  spent.   Debriefings  can  be  used  to  increase  the 


competitiveness  of  future  procurements  and  thus  allow  DOD  to 
get  more  for  each  procurement  dollar. 

Debriefing  unsuccessful  offerors  is  not  something  new. 
Debriefings  were  mentioned  as  far  back  as  1972  in  the  Report 
of  The  Commission  on  Government  Procurement,  published  in 
December  1972 .  Additionally,  debrief ing  unsuccessful  of ferors 
was  required  by  the  Defense  Acquisition  Regulation,  in  force 
before  the  advent  of  the  Federal  Acquisition  Regulation  (FAR) 
in  1984  [6:  p.D. 15 . 10- 1] .  The  FAR  states  that  debriefing  of 
unsuccessful  offerors  that  request  a  debriefing  is  required 
when  a  contract  is  awarded  on  the  basis  of  other  than  price 
alone  [7:  p. 15-41].  The  FAR  provides  some  general  guidance 
for  the  debriefings.  This  is  as  far  as  federal  laws  and 
regulations  go  in  covering  the  debriefing  process.  Army 
regulations  add  little  to  what  the  federal  regulations 
stipulate.  Details  of  who  debriefs,  when  and  where  the 
debriefing  occurs,  and  the  content  of  the  debriefing  are  left 
to  a  command's  standard  operating  procedures  or  contracting 
officers  to  decide. 

B .   OBJECTIVES 

The  objective  of  this  Thesis  is  to  examine  the  Army 
Material  Command's  process  of  debriefing  unsuccessful  offerors 
in  contracts  awarded  on  a  basis  of  other  than  price  alone. 
The  research  is  focused  on  what  the  current  debriefing  process 


is,  what  problems  there  are  with  the  current  process,  and  what 
can  be  done  to  improve  the  process . 

C.  RESEARCH  QUESTIONS 

The  following  research  questions  will  be  answered  by  the 
Thesis . 

Primary:  What  are  the  key  problems  and  issues  associated 
with  the  Army  Material  Command's  process  of  debriefing 
unsuccessful  offerors  and  what  can  be  done  to  improve  the 
process? 

Subsidiary: 

1.  What  is  the  current  debriefing  process  used  by  Army 
Material  Command? 

2 .  What  are  the  problems  associated  with  the  Army 
Material  Command's  current  debriefing  process  from  Army 
Material  Command's  perspective? 

3 .  What  are  the  problems  associated  with  Army  Material 
Command's  current  debriefing  process  from  the  unsuccessful 
offerors  perspective? 

4.  How  can  the  debriefing  process  be  modified  to  help 
Army  Material  Command  and  the  unsuccessful  offerors? 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

1.  Scope.  This  Thesis  focuses  on  the  debriefing  of 
unsuccessful  offerors  in  Army  Material  Command  (AMC) .  It 
covers  all  contracts  awarded  on  a  basis  of  other  than  price 


alone.  Five  of  the  six  major  subordinate  commodity  commands 
are  studied.  They  include  the  Armament,  Munitions  and 
Chemical  Command,  the  Aviation  rid  Troop  Command,  the 
Communications -Electronics  Command,  the  Missile  Command,  and 
the  Tank -Automotive  Command.  The  Chemical  and  Biological 
Defense  Command  is  not  looked  at  as  it  recently  reorganized 
and  just  became  part  of  AMC  in  1993.  Surveys  and  interviews 
on  the  current  process  and  how  to  improve  it  are  directed  to 
Army  Material  Command  contracting  officials  and  defense 
contractor  contracting  representatives  who  contract  with  AMC. 

2.  Limitations.   The  following  limitations  exist  in  the 
thesis . 

a.  Legislation  on  streamlining  defense  acquisition, 
which  includes  a  portion  on  debriefing  unsuccessful  offerors 
is  currently  being  discussed  in  Congress.  Uncertainty  as  to 
exactly  what  that  legislation  will  be  will  exist  until  it  is 
passed,  if  it  is  passed. 

b.  Not  all  contracting  officers  within  Army  Material 
Command  and  defense  contractors  that  work  with  Army  Material 
Command  were  contacted.  Additionally  several  were  contacted 
but  did  not  respond  to  the  survey  or  interview  request. 
Therefore  there  may  be  information  useful  to  the  thesis  that 
was  not  collected  and  thus  not  available  for  presentation  and 
analysis . 

3.  Assumptions.   The  following  assumptions  were  made  in 
this  thesis. 


a.  The   reader   of   the   thesis   has   a   general 
understanding  of  Government  contract  administration. 

b.  Current  regulations  concerning  debriefing  of 
unsuccessful  offerors  will  remain  in  effect. 

E.   DEFINITIONS  AND  ABBREVIATIONS 

1.  AMC:  Army  Material  Command.  A  major  command  in  the 
Army.  AMC  is  responsible  for  developing,  acquiring  and 
sustaining  the  material  the  Army  needs  to  fight  its  wars.  It 
has  many  subordinate  commands  that  perform  specific  missions. 

2.  AMCCOM:  Armament,  Munitions  and  Chemical  Command.  A 
major  subordinate  command  of  AMC.  AMCCOM  is  responsible  for 
the  development,  production  and  fielding  of  Army  artillery, 
mortars,  rifles,  rocket  launchers,  and  aircraft  and  tank 
armament.  It  is  also  responsible  for  acquiring  all  Army 
ammunition. 

3.  ATCOM:  Aviation  and  Troop  Command.  A  major  subordinate 
command  of  AMC.  ATCOM  is  responsible  for  overall  acquisition 
of  all  Army  aviation  assets  and  individual  equipment  for 
soldiers,  such  as  clothing,  food  and  facilities. 

4.  CBDCOM:  Chemical  and  Biological  Defense  Command.  A 
major  subordinate  command  of  AMC.  It  is  responsible  for 
research,  development,  and  production  of  biological  and 
chemical  defense  items. 

5.  CECOM:  Communication- Electronics  Command.  A  major 
subordinate  command  of  AMC.   CECOM  is  responsible  for  the 


overall  acquisition  of  Army  communications  and  electronic 
equipment . 

6.  MICOM:  Missile  Command.  A  major  subordinate  command  of 
AMC.   MICOM  acquires  all  Army  tactical  missiles  and  rockets. 

7.  MSC:   Major  subordinate  command. 

8.  SSA:  Source  Selection  Authority. 

9.  SSAC:  Source  Selection  Advisory  Council. 

10.  SSEB:  Source  Selection  Evaluation  Board. 

11.  TACOM:  Tank -Automotive  Command.  A  major  subordinate 
command  of  AMC.  TACOM  is  responsible  for  the  overall 
acquisition  of  Army  tanks,  automotive  ground  vehicles, 
construction  equipment  and  material  handling  equipment. 

F.   ORGANIZATION  OF  THE  STUDY 

Chapter  II  (background)  discusses  the  following  four  major 
areas.  The  organization  of  AMC  and  its  MSCs;  where  debriefing 
fits  in  the  acquisition  cycle;  why  debrief  unsuccessful 
offerors  and  the  laws  and  regulations  that  pertain  to 
debriefing  unsuccessful  offerors. 

Chapter  III  (research  methodology)  describes  the 
rationale  behind  the  surveys  and  details  the  interviews. 

Chapter  IV  (data  presentation  and  analysis)  presents  a 
summary  of  the  surveys  and  analysis  of  the  results,  and 
presents  and  analyzes  the  data  collected  from  the  interviews. 

Chapter  V  (conclusions  and  recommendations)  discusses  the 
conclusions  that  are  made  from  the  data  collected  and  makes 


recommendations  for  improving  the  current  debriefing  process. 
This  chapter  also  provides  areas  that  are  recommended  for 
further  study. 


II.   BACKGROUND 

A.  GENERAL 

This  chapter  will  address  the  areas  that  the  reader  must 
be  familiar  with  in  order  to  understand  the  specific 
discussion  on  debriefing  held  in  the  later  chapters.  It  will 
discuss  the  laws  and  regulations  that  pertain  to  debriefing, 
how  debriefing  fits  in  the  source  selection  process,  and  the 
reasons  why  debrief ings  are  conducted.  This  chapter  will 
also  provide  a  description  of  AMC  and  show  where  contracting 
officers  fit  in  their  organization. 

B.  LAWS  AND  REGULATIONS  ON  DEBRIEFING 
I .   Laws 

Title  10  of  the  United  States  Code  contains  the  laws 
which  pertain  to  negotiated  procurement  for  the  military. 
Currently  there  is  no  legislation  contained  in  Title  10  that 
pertains  to  debriefing  of  unsuccessful  offerors.  Title  10 
mentions  only  that  unsuccessful  offerors  be  notified  promptly 
once  contract  award  has  been  made.  This  was  added  to  Title  10 
in  1988.  [23:  sec. 2305]  This  is  not  to  say  that  Congress  is 
not  interested  in  debriefing  or  hasn't  discussed  it.  Rather, 
the  recommendation  to  pass  legislation  concerning  debriefing 
has  been  brought  up  several  times.  It  is  discussed  in  the 
Report  of  the  Commission  on  Federal  Procurement,  published  in 


10 


December  1972,  surfaced  again  in  1991  when  both  houses  of 

Congress  proposed  legislation  that  covered  debriefing,  and 

appeared  again  in  1993  as  part  of  the  Section  800  Panel  Report 

that  went  to  Congress. 

a.  Report       of        the       Commission       on       Government 
Pro curemen t 

The  Report  of  the  Commission  on  Government 
Procurement  was  conducted  in  19  72  by  a  commission  appointed  by- 
Congress  and  the  Executive  Branch.  It  consisted  of  two 
members  each  from  the  Senate,  House  of  Representatives,  and 
Executive  Branch,  one  member  from  the  Comptroller  General,  and 
five  members  from  the  public  sector.  It  was  commissioned  in 
response  to  a  concern  over  the  manner  in  which  the  federal 
procurement  process  operated  and  over  the  deficiencies  in  the 
system  [18:  p.l].  It  was  a  full  scale  study  of  the  entire 
federal  procurement  process  and  recommended  to  Congress 
methods  to  increase  the  economy,  efficiency,  and  effectiveness 
of  the  procurement  process.  One  of  those  recommendations  was 
on  post -award  policy  and  pertained  to  debriefing  of 
unsuccessful  offerors.   The  recommendation  was  that: 

When  competitive  procedures  that  do  not  involve  formal 
advertising  are  utilized,  establish  that  agencies  shall 
upon  written  request  of  an  unsuccessful  proposer, 
effectively  communicate  the  reasons  for  selecting  a 
proposal  other  than  his  own.   [1_8:  p.  25] 

The  report's  reasons  for  the  recommendation  was  that  there 

were   no   laws   requiring   communication  with  unsuccessful 

offerors   to   explain   why   their   proposals   were   not   as 
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advantageous  to  the  Government  as  the  winning  offeror's. 
Letting  an  unsuccessful  offeror  know  why  his  proposal  was  not 
as  strong  as  the  winner's  allowed  the  offeror  to  improve  and 
be  more  competitive  in  the  future.  Debriefing,  the  report 
explained,  improves  the  confidence  of  offerors  as  they  realize 
the  procurement  rules  are  being  followed  and  applied  fairly. 
Unsuccessful  offerors  thought  they  should  be  given  information 
to  understand  the  value  of  their  proposals  [18.:  p. 25]  .  When 
not  given  this  information,  protests  and  informal  complaints 
were  used  to  gain  access  to  information  as  to  why  their 
proposal  wasn't  selected  [18.:  p. 25].  Finally,  the  commission 
thought  that  by  passing  legislation  requiring  debriefing,  the 
Government  would  get  better  proposals  and  have  more 
credibility  with  competing  contractors.  Although  the 
commission  made  a  good  case  for  legislation  concerning 
debriefing  of  unsuccessful  offerors,  Congress  did  not  come 
through  with  any  appropriate  legislation. 
b.      H.R.    3161 

In  1991  the  House  of  Representatives  proposed  a 
resolution  to  amend  laws  pertaining  to  federal  procurement. 
The  resolution,  H.R.  3161,  included  a  portion  on  the 
debriefing  of  unsuccessful  offerors.  The  portion  of  the  bill 
was  introduced  over  concern  that  insufficient  debriefings  may 
actually  increase  award  protests.  [10:  p. 35]  Offerors  unsure 
of  the  fairness  of  the  award  decision  or  those  just  wanting  to 
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know  the  reasons  why  they  did  not  win  contract  award  could  use 
a  protest  as  a  tool  to  find  out  what  an  adequate  debriefing 
would  have  provided.  The  bill  was  specific  in  regard  to  what 
should  be  covered  by  statute.  The  proposed  amendment 
contained  five  sections: 

(a)  When  a  contract  is  awarded  on  a  basis  other  than  price 
alone,  unsuccessful  offerors,  upon  their  written  request, 
shall  be  debriefed  and  furnished  the  basis  for  the 
selection  decision  and  contract  award. 

(b)  In  any  procurement  for  a  contract  for  an  amount 
greater  than  $5,000,000,  a  debriefing  under  subsection  (a) 
shall  provide  to  the  offeror  requesting  the  debriefing,  at 
least  the  following  information  if  disclosure  of  such 
information  would  not  compromise  confidential  business 
information  of  the  awardee: 

(1)  the  basic  proposed  technical  solution  or 
configuration  of  the  awardee; 

(2)  the  overall  evaluated  cost  of  the  awardee  and  of 
the  debriefed  offeror; 

(3)  the  cost  or  price  associated  with  the  major 
components  of  the  awardee' s  proposal,  including  line  item 
pricing  if  practicable; 

(4)  the  overall  ranking  of  the  awardee  and  the 
debriefed  offeror,  and  the  combined  technical  and  cost 
scores  of  the  awardee  and  the  debriefed  offeror; 

(5)  the  technical  point  scores  of  the  awardee  and  the 
debriefed  offeror  on  the  factors  and  sub- factors 
identified  in  the  solicitation; 

(6)  a  description  of  the  rational  for  the  award;  and 

(7)  reasonable  responses  to  questions  posed  by  the 
debriefed  offeror  as  to  whether  source  selection 
procedures  set  forth  in  the  solicitation,  the  source 
selection  plan,  applicable  regulations,  and  other 
applicable  authorities  were  followed  by  the  Government. 

(c)  Each  solicitation  for  a  contract  greater  than 
$5,000,000  shall  notify  participating  offerors  that  the 
categories  of  information  described  in  subsection  (b)  may 
be  disclosed  by  the  Government  in  post-award  briefings. 

(d)  The  failure  of  any  agency  to  provide  a  debriefing 
satisfying  the  requirements  of  subsection  (b)  shall  be  a 
matter  subject  to  protest  to  any  forum  having  jurisdiction 
over  protests  relating  to  the  procurement,  and  the  protest 
forum,  in  its  discretion,  may  order  appropriate  relief  in 
any  such  protest. 

(e)  In  any  case  in  which  a  debriefing  is  conducted  under 
this  section  and  the  procurement  subsequently  is  resumed 
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as  a  result  of  a  successful  protest  against  award  or  other 
reason,  the  contracting  officer  shall  provide  to  each 
competing  contractor  all  information  subject  to  disclosure 
under  this  section  that  is  necessary  to  ensure  equity  and 
fair  competition  in  the  resumed  procurement."   [10:  p. 15] 

This  bill  proposed  not  only  that  debriefings  be 
conducted  by  federal  agencies  but  provided  a  detailed 
framework  for  what  should  be  included  in  those  debriefings. 
This  framework  went  far  beyond  what  the  current  regulations 
require  for  debriefings.  Debriefings  based  on  this  outline 
would  no  doubt  reduce  the  lingering  questions  left  in 
unsuccessful  offerors  minds  as  to  why  they  didn't  win  a 
contract  award.  This  outline  was  never  put  to  use  however  as 
the  bill  did  not  pass  into  law. 
c.  S.    1958 

The  Senate  introduced  a  bill  in  1991  that  was 
similar  to  the  House's  H.R.  3161.  It  was  S.  1958.  The  bill 
had  four  main  goals,  one  of  which  was  the  establishment  of  a 
meaningful  debriefing  for  unsuccessful  offerors  [19.:  p. 3]. 
The  Senate  bill  added  a  requirement  that  H.R.  3161  did  not 
contain.  A  request  for  a  debriefing  covered  under  the  Senate 
bill  must  be  made  within  ten  days  after  contract  award  and  the 
debriefing  must  take  place  within  ten  days  of  the  request 
[19 :  p. 18].  The  Senate  cited  the  need  for  legislation  on 
debriefings  as  a  result  of  debriefings  being  inconsistently 
given  and  even  when  given,  many  times  failing  to  meet  the 
objective  of  conducting  them  [19:  p. 9].   Additionally,  the 
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Senate  conceded  that  by  requiring  meaningful  debrief ings,  the 
number  of  protests  filed  as  a  result  of  no  debrief ings  or  poor 
debriefings  would  drop  [19:  p. 10].  Thus,  the  cost  and  time 
lost  due  to  a  protest  could  be  avoided  by  a  proper  debriefing. 
Industry  officials  stated  at  the  Senate  hearings  that  federal 
agencies  were  not  providing  useful  information  regarding  the 
reasons  for  award  to  the  winner  and  non- award  to  the 
unsuccessful  offerors.  They  further  stated  that  many  times 
protests  were  used  to  obtain  the  information  they  could  not 
get  at  the  debriefing  [19.:  p.  9]  .  The  Government  Accounting 
Office  confirmed  all  of  this  [19:  p. 9] .  As  was  the  case  with 
H.R.  3161  though,  S.  1958  was  not  passed  and  no  legislation  on 
debriefing  unsuccessful  offerors  resulted  from  either  bill. 
d.  Section   800  Panel  Report 

In  the  National  Defense  Authorization  Act  of  FY 
1991  Congress  directed  DOD  to  establish  an  advisory  panel  of 
Government  and  industry  experts  to  review  all  laws  affecting 
DOD  procurement .  The  panel  was  to  submit  a  report  through  the 
Secretary  of  Defense  in  January  19  9  3  that  made  recommendations 
to  streamline  defense  acquisition.  The  report  submitted  to 
Congress  is  commonly  referred  to  as  the  Section  800  Panel 
Report.  One  of  the  recommendations  that  the  panel  submitted 
was  an  amendment  for  section  2305(b) (4) (B)  of  Title  10  of  the 
United  States  Code  requiring  regulations  that  address 
debriefing  unsuccessful  offerors  [25:  p. 1-4].   The  panel's 
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recommendation  was  based  on  the  American  Bar  Association's 
19  89  Report  on  Bid  Protests  which  found  that  prompt  and 
meaningful  debrief ings  help  stem  the  tide  of  protests.  The 
Bar  Association  report  contained  the  results  of  a  survey  taken 
of  protestors  and  their  lawyers.  The  results  showed  that  a 
number  of  protests  could  be  avoided  if  timely  and  meaningful 
debrief  ings  were  conducted.  [25.:  p.  1-232]  These  debrief  ings 
would  show  that  the  procurement  was  done  properly  and  would 
provide  information  why  the  unsuccessful  offeror  was  not 
selected  for  contract  award. 

The  Section  800  Panel  thought  that  any  statute 
concerning  debriefing  unsuccessful  offerors  should  cover 
general  policy  objectives  while  the  regulations  should  cover 
the  details.  The  panel  recommended  that  the  statute  require 
three  things  of  the  regulations.  First,  the  regulations 
should  set  up  criteria  for  determining  when  a  debriefing  is 
required.  The  panel  recognized  that  debrief ings  are  not 
essential  for  such  actions  as  small  purchases,  and  procurement 
actions  where  there  are  no  significant  evaluation  areas  other 
than  cost.  Next  the  panel  believed  that  regulations  must  be 
required  to  ensure  that  debrief ings  be  held  within  15  calendar 
days  after  award,  whenever  possible.  The  panel  believed  that 
the  more  timely  a  debrief  is  held  the  less  likely  it  was  that 
a  protest  would  be  filed.  Lastly,  the  panel  believed  the 
regulations  must  require  the  debriefing  to  include  the 
strengths   and  weaknesses   of   the  unsuccessful   offeror's 
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proposal.  By  providing  strengths  and  weaknesses  the  panel 
thought  an  unsuccessful  offeror  would  better  understand  why  he 
was  not  the  winner  and  would  thus  be  less  inclined  to  file  a 
protest.  [25:  p. 1-69]  The  outcome  of  the  Section  800  Report 
and  it's  recommendations  on  debriefing  are  yet  to  be 
determined.  The  report  was  presented  to  Congress  in  October 
1993  and  is  still  being  discussed. 

e.      Federal  Acquisition  Regulation 

The  Federal  Acquisition  Regulation  (FAR)  is  a  set 
of  regulations  that  cover  the  acquisition  of  all  goods  and 
services  for  all  federal  agencies.  The  FAR  covers  debriefing 
of  unsuccessful  offerors  in  15.1003  which  provides  that: 

(a)  When  a  contract  is  awarded  on  the  basis  of  other  than 
price  alone,  unsuccessful  offerors,  upon  their  written 
request,  shall  be  debriefed  as  soon  as  possible  and 
furnished  the  basis  for  the  selection  decision  and 
contract  award. 

(b)  Debriefing  information  shall  include  the  Government's 
evaluation  of  the  significant  weak  or  deficient  factors  in 
the  proposal;  however,  point -by-point  comparisons  with 
other  offerors'  proposals  shall  not  be  made.  Debriefing 
shall  not  reveal  the  relative  merits  or  technical  standing 
of  competitors  or  the  evaluation  scoring.  Moreover, 
debriefing  shall  not  reveal  any  information  that  is  not 
releasable  under  the  Freedom  of  Information  Act;  for 
example - 

(1)  Trade  secrets; 

(2)  Privileged  or  confidential  manufacturing  processes 
and  techniques;  and 

(3)  Commercial  and  financial  information  that  is 
privileged  or  confidential,  including  cost  breakdowns, 
profit,  indirect  cost  rates,  and  similar  information. 

(c)  The  contracting  officer  shall  include  a  summary  of  the 
debriefing  in  the  contract  file.  [7:  p. 15-41] 

It  is  important  to  note  what  information  the  FAR 

allows  the  debriefing  agency  to  release.    Companies  that 
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request  debriefings  are  generally  looking  for  as  much 
information  as  possible  to  explain  why  their  proposals  were 
not  selected  for  contract  award.  The  FAR  does  allow  general 
comparisons  to  be  made  between  proposals.  However,  the 
debriefing  will  not  cover  the  relative  merits  or  technical 
standing  of  competitors  or  the  evaluation  scoring.  [7:  p.  15- 
41]  A  definition  of  relative  merits,  technical  standing  and 
evaluation  standing  is  needed  as  these  terms  can  be 
interpreted  differently  by  different  people.  Dr.  David  Lamm 
of  the  Naval  Postgraduate  School  describes  what  these  three 
terms  mean.  Relative  merits  refer  to  the  approaches  taken  by 
an  offeror  in  his  attempt  to  satisfy  the  evaluation  criteria 
of  the  RFP.  This  includes  the  evaluated  quality  of  those 
approaches.  Technical  standing  is  the  ranking  of  the 
technical  area  or  areas  with  respect  to  other  offerors.  It 
could  be  in  the  form  of  scores  or  numerical  ranking.  Lastly, 
evaluation  scoring  is  the  numerical  score  given  to  each  area 
and  total  numerical  score  of  all  areas.  [11]  Thus,  the  FAR 
emphasizes  the  discussion  be  on  the  debriefed  offeror's 
proposal  and  discourages  any  discussion  on  other  competitor's 
proposals . 

f.      Army  FAR  Supplement 

The  Army  FAR  Supplement  (AFARS)  provides 
additional  guidance  to  AMC  for  debriefing  unsuccessful 
offerors.   It  consists  of  three  sections  which  provide  that: 
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a.  The  contracting  officer  will  release  to  unsuccessful 
offerors  the  required  information  as  prescribed  in  FAR 
15.1001. 

b.  When  debrief  ings  are  requested  they  shall  be  in 
accordance  with  FAR  15.1003. 

(1)  Debriefings  will  be  with  only  one  offeror  at  a  time 
and  will  not  be  conducted  until  after  contract  award. 

(2)  The  debriefing  must  be  confined  to  a  discussion  of 
the  offeror's  proposal  and  its  advantages  and 
disadvantages  in  relation  to  the  requirements  of  the  RFP. 
Comparisons  should  not  be  made  relative  to  the  proposal  of 
other  offerors. 

(3)  No  information  will  be  disclosed  to  an  offeror  as 
to  the  weights  or  ratings  assigned. 

c.  Normally,  the  debriefings  should  be  conducted  by  or 
under  the  direction  of  the  contracting  officer.  It  should 
be  noted  that  some  contractors  may  wish  to  discuss  the 
outcome  at  higher  levels  of  authority.  Although  this 
should  not  be  encouraged,  neither  should  barriers  be 
placed  in  their  way  of  getting  an  explanation  from 
responsible  officials.  [1:  p. 10-3] 

This  places  further  restrictions  on  the  debriefing 

process.    Of  particular  note  is  that  the  AFARS  prevents 

general  comparisons  with  other  offerors'  proposals  even  though 

such  comparisons  are  allowed  by  the  FAR.   The  AFARS  also 

states  that  debriefings  should  be  limited  to  the  offeror's 

proposal.    This  prevents  any  discussion  of  the  winning 

offeror's  proposal.    However,  the  agency  that  wrote  this 

guidance  stated  that  each  debriefing  should  be  treated  on  a 

case  by  case  basis  and  that  contracting  commands  have  the 

latitude  to  deviate  from  this  guidance.    The  reason  this 

guidance  was  written  was  that  there  had  been  problems  with 

unsuccessful  offerors  getting  some  information  on  the  winner's 

proposal  evaluation  and  then  trying  to  re -engineer  the  source 

selection  evaluation  process.   With  the  limited  information 

that  some  unsuccessful  offerors  received  they  were  sometimes 
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coming  up  with  the  conclusion  that  they  couldn't  have  lost  the 

procurement  competition  and  then  protested  the  procurement. 

The  unsuccessful  offeror  will  never  understand  the  complete 

selection  process  because  proprietary  business  data  cannot  be 

released.   Therefore  he  will  never  have  a  complete  picture  of 

the  source  selection  process.  [20.]    Additionally,  the  AFARS 

stipulates  that   contracting  officers  debrief  one  offeror  at 

a  time  and  only  after  contract  award.   The  AFARS  highlights 

the  contracting  officer's  role  as  the  official  interface 

between  his  command  and  the  defense  contractor  and  it  requires 

that  the  debriefing  should  be  conducted  or  directed  by  him. 

g.      AMC-P   715-3    Vol.    1 

AMC  has  it's  own  publication  which  discusses 

source  selection  and  it  contains  a  section  on  debriefing. 

AMC-P  715-3,  Vol.  1  reiterates  the  same  guidance  that  the 

AFARS  provides  except  it  adds  that : 

.  .  .  normally  the  Debrief ings  should  be  conducted  by  or 
under  the  direction  of  the  [contracting  officer] ,  although 
the  manner  in  which  debriefings  are  to  be  handled  is 
discretionary  with  the  SSA.  [2:  p. 51] 

This  reinforces  the  role  the  SSA  has  in  the  source  selection 

process   to   include   the   last   communications   with   the 

unsuccessful  offerors,  that  of  the  debriefing.   The  SSA  can 

influence  the  debriefing  in  ways  the  contracting  officer 

cannot.  The  SSA  can  provide  resources  for  the  debriefing  that 

may  not  be  available  without  his  help.   This  is  especially 
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true  for  SSEB  members  who  can  improve  the  debriefing  by- 
participating  in  it. 

C.   WHY  DEBRIEF  UNSUCCESSFUL  OFFERORS 

There  are  several  reasons  why  the  Army  should  debrief 
unsuccessful  offerors.  The  first  reason  is  because  it  is 
required.  The  FAR  states  that  when  a  contract  is  awarded  on 
a  basis  of  other  than  price  alone  unsuccessful  offerors  will, 
upon  written  request,  be  debriefed  as  soon  as  possible  and  be 
provided  the  basis  for  the  selection  decision  [7:  p. 15-41]. 
The  FAR  makes  no  mention  of  the  size  or  complexity  thresholds 
that  need  to  be  met  before  debrief ings  will  be  held.  It 
merely  states  that  if  one  or  more  factors,  other  than  price, 
is  used  to  determine  the  source  selection,  then  unsuccessful 
offerors  are  entitled  to  a  debriefing  when  they  ask  for  one  in 
writing.  Thus,  all  federal  agencies  are  required  to  debrief 
all  unsuccessful  offerors  that  request  a  debriefing  in 
writing. 

Another  reason  to  debrief  is  to  help  improve  the  future 
competitiveness  of  the  unsuccessful  offeror.  By  providing  the 
unsuccessful  offerors  with  the  rationale  behind  why  they 
weren't  selected  for  contract  award  they  will  be  able  to 
submit  better  proposals  for  future  contracts  [17:  p. 528]. 
Commenting  on  all  evaluated  areas  and  identifying  strengths 
and  weaknesses  of  the  offeror's  proposal  will  allow  him  to 
capitalize  on  his  strong  areas  and  improve  on  his  weaknesses 
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for  future  proposals.  This  should  improve  his  competitiveness 
for  any  subsequent  solicitations  he  responds  to.  Without  this 
explanation  the  unsuccessful  offeror  may  make  false 
assumptions  about  what  improvements  he  needs  to  make  and  thus 
may  continue  to  commit  the  same  costly  errors  on  future 
proposals.  Improving  his  competitive  position  also  helps  the 
buyer  as  it  receives  a  more  competitive  proposal  in  future 
procurements . 

Fairness  is  another  reason  why  the  unsuccessful  offeror 
should  be  debriefed  [3.:  p.l]  .  The  contractor  worked  hard  on 
getting  his  proposal  done  and  spent  precious  company  resources 
in  the  process.  For  large  solicitations  his  best  efforts  to 
win  the  contract  award  may  result  in  months  of  work  and 
hundreds  of  thousands  of  dollars  spent  preparing  a  proposal. 
The  interests  of  fairness  dictate  that  the  contractor  get  some 
feedback  as  to  why  he  wasn't  selected  for  contract  award.  It 
is  only  fair  that  federal  agencies  respond  to  the  contractors 
efforts  and  spend  some  time  explaining  how  they  evaluated  the 
proposal . 

Next,  debrief ings  can  show  unsuccessful  offerors  that  the 
source  selection  decision  was  rational  and  conformed  to  the 
requirements  and  evaluation  criteria  set  out  in  the  RFP 
[17:  p. 528] .  With  no  feedback  the  unsuccessful  offeror  would 
less  likely  know  for  sure  if  the  source  selection  was  based 
solely  on  the  evaluation  criteria  in  the  RFP  and  if  the 
evaluation  board  used  the  criteria  in  a  logical  way  to 
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evaluate  it's  proposal.  By  providing  a  debriefing  the 
unsuccessful  offeror  can  see  that  the  evaluation  committee  did 
indeed  use  all  the  criteria  identified  in  the  RFP  and 
evaluated  each  area  in  accordance  with  the  standards  set  forth 
for  each  area. 

Lastly,  by  providing  a  debriefing  contracting  officials 
should  reduce  the  possibility  of  protests  being  filed  which 
could  delay  the  procurement  [3.:  p.l]  .  By  placing  their  "cards 
on  the  table"  unsuccessful  offerors  can  see  that  the  contract 
agency  has  nothing  to  hide  and  is  acting  in  good  faith. 
Additionally,  they  could  get  the  information  they  are  looking 
for  to  find  out  why  they  lost  without  having  to  file  a 
protest.  Thus,  a  protest  would  less  likely  be  filed  as  a 
result  of  providing  a  debriefing.  This  is  the  reason  the 
Section  800  Panel  cited  as  it's  basis  for  requiring 
debrief ings . 

D.   WHERE  DEBRIEFING  FITS  IN  THE  SOURCE  SELECTION  PROCESS 

The  debriefing  process  fits  in  near  the  end  of  the  source 
selection  process  but  is  not  an  isolated  event.  Rather,  all 
steps  in  the  process  have  some  effect  on  the  debriefing  of 
unsuccessful  offerors. 

The  source  selection  process  begins  with  the  development 
of  the  source  selection  plan  (SSP) ,  which  is  normally 
comprised  of  two  parts  dealing  with  the  source  selection  team 
membership  and  the  evaluation  criteria.   The  first  part  covers 


23 


the  details  of  the  organization,  membership  and 
responsibilities  of  the  source  selection  party.  It  describes 
who  is  on  the  Source  Selection  Advisory  Council  (SSAC)  and 
Source  Selection  Evaluation  Board  (SSEB) ,  if  it  is  used.  The 
advisory  council's  role  is  to  advise  the  SSA  on  the  source 
selection  process  and  actual  source  selection.  The  SSA  is  the 
person  who  is  responsible  for  the  entire  source  selection 
process.  Two  of  his  important  duties  are  to  ensure  the  source 
selection  plan  is  properly  written  and  executed  and  to  select 
the  winning  proposal  based  on  the  information  the  SSEB  and 
SSAC  give  him  [5:  p.lO-B-2].  The  SSAC  may  provide  the  SSA 
with  a  comparative  analysis  of  the  evaluation  results.  Care 
must  be  taken  to  select  personnel  for  the  advisory  council  who 
have  the  appropriate  skills  and  experience  to  properly  advise 
the  SSA.  The  SSEB  is  responsible  to  evaluate  all  proposals  in 
accordance  with  the  evaluation  criteria  set  in  the  SSP  and 
RFP.  The  SSEB  members  must  have  the  proper  experience  and 
expertise  to  fully  understand  the  evaluation  criteria  and  make 
competent  evaluations  of  proposals  in  accordance  with  those 
criteria.  Selection  of  the  right  people  to  fill  these 
positions  will  result  in  the  thorough  evaluation  of  each 
proposal  and  the  proper  results  given  to  the  SSA.  This  in 
turn  will  result  in  the  correct  proposal  receiving  contract 
award  and  meaningful  comments  on  the  strengths  and  weaknesses 


24 


of  all   other  proposals   for  debriefing  of  unsuccessful 
offerors . 

The  second  part  of  the  source  selection  plan  covers  the 
evaluation  criteria.  The  SSA  and  SSAC  are  responsible  for  the 
development  and  weighting  of  the  evaluation  criteria.  The 
criteria  are  used  by  the  SSEB  to  evaluate  each  proposal.  All 
proposals  are  evaluated  in  the  same  manner  to  ensure  a 
consistent  process.  The  evaluation  criteria  must  be  selected 
and  written  to  ensure  the  Government  gets  the  product  that 
best  meets  the  solicitation  and  statement  of  work. 
Establishing  a  method  to  evaluate  each  criteria  is  extremely 
important  as  it  serves  as  the  basis  for  the  evaluation  and 
justification  of  each  rating.  Evaluation  criteria  are  to  be 
measured  quantitatively  and  qualitatively  and  must  be  written 
as  such.  There  should  be  a  standard  against  which  each 
criteria  can  be  judged.  Additionally,  the  evaluation  criteria 
must  be  properly  weighted  to  reflect  the  relative  importance 
of  each  criteria  as  determined  by  the  SSA  and  SSAC. 
Incomplete  or  poorly  worded  criteria  or  improperly  weighted 
evaluation  criteria  can  result  in  an  inaccurate  assessment  of 
proposals.  Problems  with  source  selection  and  subsequent 
debrief ings  would  then  follow.  Narrative  comments  extracted 
from  the  evaluation  for  use  in  a  debriefing  may  then  be 
incorrect,  misleading  or  incomplete.    This  results  in  a 
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debriefing  that  is  inaccurate  or  lacks  any  specifics.  These 
narrative  comments  are  crucial  to  a  successful  debriefing. 
They  provide  the  concise  information  unsuccessful  offerors 
need  to  understand  what  their  strengths  and  weaknesses  were  in 
each  area.  Establishing  comprehensive  and  accurate  standards 
for  evaluation  criteria  and  weighting  them  properly  will 
prevent  any  problems  evaluating  proposals  and  instead  yield 
correct  and  meaningful  evaluations  and  comments  that  can  be 
used  to  make  a  correct  source  selection  and  improve  the 
debriefing. 

Next,  a  competitive  range  is  determined.  This  is  normally 
determined  by  the  contracting  officer  and  approved  by  the  SSA. 
Three  areas  are  looked  at  when  evaluating  proposals  to 
determine  if  they  are  in  the  competitive  range.  First,  the 
proposal  must  be  determined  to  be  responsive.  In  the  terms  of 
negotiated  procurement  the  proposal  is  considered  to  be 
responsive  unless  the  offeror  refuses  to  comply  with  any  of 
the  requirements  of  the  RFP.  The  other  two  areas  to  be  looked 
at  are  the  technical  merits  and  cost  of  the  proposal. 
Proposals  are  to  be  deemed  in  the  competitive  range  with 
regard  to  these  two  areas  as  long  as  there  is  a  possibility 
that  the  proposal  can  win  the  contract  award  [9:  p.4-9]  .  When 
a  proposal  falls  out  of  the  competitive  range  the  offeror  is 
notified.  Debrief ings  resulting  from  being  excluded  from  the 
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competitive  range  may  not  be  as  detailed  as  debriefings  given 
to  offerors  who  make  it  all  the  way  through  to  best  and  final 
offers  (BAFO's)  .  A  comprehensive  evaluation  may  not  have  been 
needed  to  determine  the  proposal  was  outside  the  competitive 
range.  Discussions  are  held  with  those  offerors  in  the 
competitive  range.  Meaningful  discussions  should  have  a 
direct  impact  on  the  debriefing  of  unsuccessful  offerors. 
Meaningful  discussions  will  identify  any  deficient  areas  in  a 
proposal  and  help  clarify  any  questions  the  evaluation  team 
has  with  respect  to  the  proposal .  This  allows  the  contractor 
to  make  modifications  to  his  proposal  to  increase  it's 
competitiveness.  Additionally,  productive  discussions  should 
head  off  any  surprises  in  the  debriefing  regarding  weak  areas 
in  the  proposal  as  these  weaknesses  would  have  already  been 
identified  and  then  addressed  by  the  offeror. 

BAFO's  are  received  after  discussions  are  concluded.  At 
this  time  all  offerors  still  in  the  competitive  range  may 
submit  final  changes  to  their  proposals  and  make  a  final 
offer.  A  final  evaluation  is  done  on  all  those  still  in  the 
competition.  BAFO's  may  change  the  evaluation  of  those  areas 
amended  and  thus  influence  the  award  and  subsequent 
debriefing  comments  of  those  areas. 

Next,  a  source  selection  decision  is  made,  contract  award 
is   completed   and   unsuccessful   offerors   are   notified. 
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Information  given  out  during  notification  includes  the  number 
of  RFP's  sent  out  and  proposals  received,  the  contract  price, 
and,  in  general  terms,  the  reason  why  the  offeror's  proposal 
was  not  the  winning  proposal  [7:  p. 15-41]  This  concludes  the 
source  selection  team's  interaction  with  the  unsuccessful 
offeror,  unless  they  take  part  in  the  debriefing  or  a  protest 
is  filed. 

For  solicitations  that  do  not  require  a  formal  source 
selection  team  the  contracting  officer  plays  a  bigger  role  in 
the  process.  Evaluation  criteria  may  be  written  by  the 
contracting  officer  and  a  technical  and  cost  analyst.  The 
proposal  evaluations  may  be  done  solely  by  the  contracting 
officer  or  may  involve  a  small  team  of  evaluators,  such  as  a 
technical  evaluator  and  cost  analyst.  Even  though  the  process 
is  smaller  and  less  complicated  the  impact  that  each  part  of 
the  source  selection  process  has  on  debriefing  unsuccessful 
offerors  is  the  same. 

E.   ARMY  MATERIAL  COMMAND 

AMC  is  an  Army  major  command  headed  by  a  four  star 
general.  It's  basic  mission  is  to  develop,  buy  and  maintain 
material  for  the  Army  [24:  p. 2]  .  It  is  composed  of  depots, 
laboratories,  arsenals,  manufacturing  and  maintenance 
facilities,  proving  grounds,  test  ranges,  and  buying  offices. 


28 


There  are  currently  six  commodity  commands  and  four  functional 
commands  within  AMC.   These  units  are  responsible  for  the 
research,  development,  testing,  acquisition,  and  maintenance 
of  the  Army's  material  resources. 
1.   Functional  Commands 

The  four  functional  commands  include  the  Army  Research 
Laboratory  (ARL) ,  the  Depot  System  Command  (DESCOM) ,  the 
Simulation  Training  and  Instrumentation  Command  (STRICOM)  ,  and 
the  Test  and  Evaluation  Command  (TECOM) . 

a.  ARL 

The  ARL  is  a  command  composed  of  research  centers 
that  generate  new  technologies  and  advanced  concepts  for  all 
existing  and  future  Army  equipment. 

b.  DESCOM 

DESCOM  is  responsible  for  the  operation  of  the 
Army's  depots.  The  depots  receive,  store,  issue,  perform 
maintenance  on  and  dispose  of  Army  equipment . 

c.  STRICOM 

STRICOM' s  mission  is  to  manage  and  direct  the 
Army's  simulation,  training  and  test  instrumentation  needs. 
In  addition  it  does  the  same  for  the  new  Distributed 
Interactive  Simulation  technology  that  the  Army  is  using. 
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d.  TECOM 

TECOM  is  the  command  responsible  for  the  planning 
and  test  and  evaluation  of  the  Army's  equipment.    This 
includes  managing  all  testing  facilities  and  equipment. 
2.   Commodity  Commands 

The  six  commodity  commands  include  the  Armament, 
Munitions,  and  Chemical  Command  (AMCCOM) ,  the  Aviation  and 
Troop  Command  (ATCOM) ,  the  Communications -Electronics  Command 
(CECOM) ,  the  Missile  Command  (MI COM) ,  the  Tank -Automotive 
Command  (TACOM) ,  and  the  Chemical  and  Biological  Defense 
Command  (CBDCOM) .  Each  of  these  MSC's  provide  research  and 
development,  engineering,  procurement,  and  logistics  support 
to  their  commodity  areas.  Additionally  they  provide  matrix 
support  to  the  Program  Executive  Officers  of  their  respective 
commodity  areas.  These  MSC's  do  almost  all  of  the  acquisition 
actions  within  AMC.  Together  they  spent  over  95  percent  of 
AMC's  acquisition  dollars  in  1993  [15]. 
a .  AMCCOM 

AMCCOM  is  responsible  for  armament,  ammunition  and 
chemical  defense  equipment.  Armament  includes  towed  and  self 
propelled  artillery,  aircraft  and  tank  armament,  mortars,  fire 
control  systems,  gun- type  air  defense  weapons,  and  infantry 
weapons.  Ammunition  includes  all  ordnance  for  weapons  systems 
from  small  arms  infantry  weapons  to  large  artillery  and  tank 
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rounds.  Other  items  include  bombs,  rocket  and  missile 
warheads,  fuzes,  mines,  grenades,  pyrotechnics,  smoke,  and 
incendiary  devices.  The  AMCCOM  organization  includes  the 
Research,  Development  and  Engineering  Center,  three  arsenals, 
15  active  ammunition  plants,  and  Acquisition  Center. 
b.      ATCOM 

ATCOM  is  responsible  for  aviation  assets  and  troop 
support  items.  Aviation  assets  include  Army  aircraft, 
aircraft  engines,  spare  parts,  air  traffic  control  hardware 
and  aviation  ground  support  equipment.  Troop  support  items 
include  all  clothing  and  protective  clothing  items,  footwear, 
food,  food  service  equipment,  water  supply  equipment, 
petroleum  distribution  equipment,  all  aerial  delivery  systems 
such  as  parachutes  and  slings,  generators,  Army  watercraft, 
and  rail  and  bridging  equipment.  Major  activities  in  ATCOM 
include  three  research,  development  and  engineering  centers, 
a  weapon  system  management  center,  an  integrated  material 
management  center  and  an  acquisition  center. 
C  CBDCOM 

CBDCOM  is  responsible  for  the  commodity  management 
of  all  chemical  and  biological  related  items.  This  includes 
protective  suits,  footwear  and  masks,  detecting  and  monitoring 
equipment,  and  decontamination  equipment  and  materials.  Major 
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elements  of  CBDCOM  are  a  research,  development  and  engineering 
center  and  acquisition  center. 

d.  CECOM 

CECOM  is  responsible  for  all  Army  communications 
and  electronics  equipment.  Items  include  radios  and  radio 
related  equipment,  electronic  warfare  items,  photographic 
equipment,  radars,  automated  data  processing  and  management 
information  systems  items,  batteries,  ground  and  airborne 
surveillance  items,  night  vision  equipment,  and  satellite 
systems.  Major  activities  of  CECOM  include  the  Research, 
Development  and  Engineering  Center,  the  Command,  Control, 
Communications,  and  Intelligence  Logistics  and  Readiness 
Center  and  the  Acquisition  Center. 

e .  MICOM 

MICOM  is  responsible  for  all  equipment  related  to 
rocket  and  missile  systems.  This  includes  all  components  and 
subsystems  related  to  air-to-ground  missile  systems,  surface- 
to-surface  missile  systems,  surface-to-air  missile  systems, 
ballistic  missiles,  and  anti-armor  missile  systems.  Major 
units  in  MICOM  include  a  research,  development  and  engineering 
center,  an  integrated  management  center  and  acquisition 
center. 
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f.       TACOM. 

TACOM  is  responsible  for  commodity  management  of 
all  combat  and  tactical  vehicles.  This  includes  tanks,  all 
carriers  to  include  those  for  personnel,  cargo,  and  missile, 
self  propelled  artillery  vehicles,  wheeled  vehicles,  special 
purpose  equipment,  trailers,  construction  equipment,  and 
materials  handling  equipment.  Major  elements  of  TACOM  are  the 
Research,  Development  and  Engineering  Center,  Integrated 
Material  Management  Center,  Weapon  Systems  Management  Center, 
and  Acquisition  Center. 

F.   CONTRACTING  OFFICERS  IN  THEIR  ORGANIZATION 

The  AMC  MSC's  are  each  composed  of  several  major  centers. 
Centers  common  to  the  MSC's  include  the  Research,  Development, 
and  Engineering  Center,  the  Integrated  Material  Management  or 
Logistics  Center,  and  the  Acquisition  Center.  The 
contracting  officers  that  procure  equipment  and  services  are 
located  in  the  Contract  Operations  Directorate  of  the 
Acquisition  Center.  The  Acquisition  Center  is  composed  of 
several  directorates.  These  are  the  Contract  Operations, 
Acquisition  Policy,  Production  Management,  and  Quality 
Management  Directorates.  The  Contract  Operations  Directorate 
is  split  up  into  sections  by  commodity  line  and  contracting 
officers  are  assigned  to  each  section. 
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G.   SUMMARY 

This  chapter  identified  the  laws  and  regulations  that 
cover  debriefing  unsuccessful  offerors,  the  reasons  to 
debrief,  where  debriefing  fits  in  the  source  selection 
process,  what  AMC  is,  and  where  contracting  officers  fit  in 
their  organization.  An  understanding  of  these  topics  will 
allow  the  reader  to  better  understand  how  debriefing 
procedures  can  be  improved. 
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III.   METHODOLOGY 

A.   GENERAL 

The  methods  used  in  the  Thesis  to  collect  information  and 
data  include  a  literature  review,  two  survey  questionnaires, 
and  personal  interviews.  The  literature  search  was  done  first 
to  collect  written  material  concerning  debriefing  unsuccessful 
offerors.  This  material  and  a  personal  interview  with  a 
Principle  Assistant  Responsible  for  Contracting  (PARC)  were 
used  to  develop  the  two  survey  questionnaires.  Next,  surveys 
were  sent  out  to  collect  data  concerning  the  debriefing 
process.  One  survey  was  developed  for  defense  contractors. 
The  other  was  written  for  AMC  contracting  officers.  The  two 
surveys  were  very  similar  in  nature  and  were  geared  to  illicit 
responses  to  almost  identical  questions  from  the  two  sides 
participating  in  a  debriefing.  Personal  interviews  were 
conducted  with  defense  contracting  officials  to  verify  the 
results  of  the  survey  and  as  a  follow  up  to  problem  areas 
identified  in  the  surveys  that  warranted  further  exploration. 
Finally,  personal  interviews  were  done  with  AMC  contracting 
officers.  These  were  done  to  verify  survey  results  and  to 
respond  to  potential  solutions  to  problem  areas  that  were 
identified  in  the  preceding  stages. 
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B.   LITERATURE  SEARCH 

Literature  research  provided  a  background  of  information 
on  debriefing.  This  information  was  then  used  as  a  framework 
for  the  rest  of  the  thesis  research.  It  was  used  to  develop 
the  surveys  and  personal  interview  questions.  Literature 
research  started  with  a  search  of  the  Naval  Postgraduate 
School  Library,  Stanford  Business  School  Library,  the  Defense 
Logistics  Information  Exchange,  and  Defense  Technical 
Information  Center.  No  major  publications  on  debriefing  were 
found  as  a  result  of  this  search.  Instead,  several  works  had 
small  sections  that  pertained  to  debriefing.  This  provided 
only  a  small  amount  of  information  concerning  debriefing. 
Next,  a  search  was  done  on  Government  publications  and 
Congressional  documents.  This  search  resulted  in  two 
Government  documents  that  discussed  debriefing  procedures. 
These  are  the  Report  of  the  Commission  on  Government 
Procurement .  and  Streamlining  Acquisition  Laws.  Additionally, 
two  Congressional  reports  were  found.  They  are  H.R.  3161,  and 
S.  1958,  which  both  represent  Congressional  attempts  to  pass 
legislation  on  the  debriefing  process.  Lastly,  the  FAR  and 
DOD  regulations  and  instructions  that  pertained  to  debriefing 
unsuccessful  offerors  were  collected.  This  provided  the 
specific  information  regarding  the  current  debriefing  process 
within  AMC. 
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C.   SURVEY  QUESTIONNAIRES 

1.  General 

Two  survey  questionnaires  were  designed  and  utilized 
to  gather  data  on  AMC's  debriefing  procedures.  One  survey  was 
constructed  for  defense  contractors  and  the  other  for  AMC 
contracting  officials. 

2 .  Target  Audience 

a.  Defense  Contractor  Survey 

The  target  audience  for  the  survey  for  defense 
contractors  was  a  broad  cross -section  of  defense  contractors 
that  contract  with  AMC.  This  selection  of  defense  contractors 
included  large  companies,  small  companies,  small  and 
disadvantaged  companies,  service  companies,  engineering 
companies,  hardware  companies,  and  software  companies. 
Additionally,  the  selection  of  companies  was  done  to  ensure 
that  all  AMC  MSC's  were  represented  in  the  survey  target 
audience. 

The  companies  were  selected  from  a  list  of  defense 
contractors  that  appeared  in  the  May/June  1993  edition  of 
National  Defense  magazine.  Several  small  companies,  and  small 
and  disadvantaged  companies  were  also  selected  from  the  MICOM 
source  solicitation  mailing  list  to  ensure  an  adequate 
representation  of  these  companies.  Initially,  a  list  of 
companies  to  send  surveys  to  was  being  compiled  by  calling  AMC 
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MSC's  and  asking  for  names  of  companies  that  were  recent 
unsuccessful  offerors.  However  this  method  was  abandoned 
because  of  a  concern  that  the  MSC's  may  have  been  providing 
names  of  "happy"  unsuccessful  offerors  and  not  giving  names  of 
companies  that  were  "unhappy"  with  the  debriefing  they 
received.  This  method  of  developing  a  survey  mailing  list 
would  most  likely  have  resulted  in  skewed  survey  results  by 
understating  any  problems  with  the  debriefing  process. 
Therefore,  the  author  used  the  National  Defense  magazine  list 
of  defense  contractors,  supplemented  by  the  MICOM  source 
solicitation  mailing  list  to  get  a  random  list  of  defense 
contractors.  This  would  result  in  more  objective  survey 
results . 

Surveys  were  sent  to  the  director  of  contracts  of 
each  company.  Instructions  on  the  survey  requested  that 
personnel  familiar  with  debriefings  from  officials  within  AMC 
and  who  received  a  debriefing  from  an  AMC  MSC  within  the  last 
five  years  complete  the  survey.  Surveys  were  sent  to  76 
companies  with  a  total  of  204  surveys  being  mailed.  Each 
company  was  sent  at  least  two  surveys.  Ten  companies  were 
contacted  by  phone  before  surveys  were  sent  to  them  to  ensure 
that  the  surveys  would  be  answered  and  returned.  This  was 
done  to  ensure  a  sample  of  surveys  mailed  would  be  returned 
for  analysis.  These  companies  indicated  the  number  of  surveys 
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that  they  wanted  mailed  to  them.  Of  the  204  surveys  mailed 
out,  41  were  answered  and  returned,  another  15  surveys  were 
returned  that  were  not  answered  with  an  explanation  that  the 
company  either  had  not  had  a  debriefing  from  an  official 
within  AMC  within  the  last  five  years  or  they  had  not  bid  on 
any  AMC  contracts  within  the  last  five  years.  The  goal  for 
the  number  of  completed  surveys  to  be  returned  was  30.  This 
is  the  number  of  samples  needed  to  approximate  the  mean  of  the 
entire  survey  population  according  to  statistics  textbooks 
[2_6:  p. 298]  .  Even  though  statistical  analysis  was  not  the 
design  of  this  survey,  enough  survey  results  were  desired  to 
ensure  an  approximation  of  results  that  the  entire  population 
of  defense  contractors  that  contract  with  AMC  would  have 
yielded. 

Jb.  AMC  Contracting  Officer  Survey 

The  target  audience  for  the  AMC  contracting 
officer  survey  was  contracting  officials  within  the  AMC 
commodity  commands  that  conduct  debriefings  for  unsuccessful 
offerors.  AMC  MSC's  were  contacted  and  Acquisition  Center 
Chiefs  and  Directorate  Chiefs  provided  the  names  of 
contracting  officers  within  each  command  that  conduct 
debriefings.  A  representative  sample  of  contracting  officials 
was  provided  for  each  of  the  five  MSC's  studied  in  this 
thesis.  Surveys  were  sent  to  the  42  contracting  officials  who 
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names  were  provided  by  their  MSC.   Responses  were  received 
from  32  officials.   The  goal  was  to  get  3  0  responses. 
3 .   Survey  Design 

The  two  surveys  are  almost  identical  in  nature  as  all 
but  three  questions  are  the  same.  The  surveys  were  designed 
to  provide  data  on  the  respondent's  views  of  the  AMC 
debriefing  process.  This  two  survey  method  is  being  used  to 
compare  answers  between  the  two  groups  to  see  where 
similarities  and  differences  in  opinion  exist.  They  were 
tailored  to  yield  sufficient  information  on  the  debriefing 
process  while  at  the  same  time  being  short  enough  to  ensure  an 
adequate  number  of  surveys  were  completed  and  returned.  A 
copy  of  each  survey  appears  in  the  appendix.  The  surveys  have 
two  major  types  of  questions.  The  first  type  consists  of 
multiple  choice  and  specific  fill  in  the  blank  questions  and 
the  second  type  consists  of  general  open  ended  questions  on 
the  overall  debriefing  process. 

The  first  type  of  questions  was  designed  to  get 
responses  to  specific  questions  on  the  debriefing  process. 
The  defense  contractor  survey  consists  of  22  multiple  choice 
questions  and  two  specific  fill  in  the  blank  questions.  The 
Survey  for  AMC  officials  has  21  multiple  choice  questions  and 
three  specific  fill  in  the  blank  questions.  Possible  answers 
on  the  multiple  choice  questions  range  from  two  to  four  with 
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the  exception  of  one  question  which  had  six  possible  answers. 
These  possible  answers  were  chosen  in  order  to  categorize 
responses  to  aid  in  analyzing  the  survey  data.  The  multiple 
choice  questions  each  have  a  blank  space  for  respondents  to 
amplify  their  answers  or  make  any  comments  in  reference  to  the 
questions  or  their  answers.  The  comments  were  to  be  used  to 
aid  in  analyzing  the  answers  and  to  see  if  any  comments 
appeared  repeatedly  throughout  the  returned  surveys . 

The  second  type  of  questions  consisted  of  open  ended 
questions.  The  defense  contractor  survey  had  five  open  ended 
questions  while  the  AMC  survey  had  four.  The  objective  of 
using  these  questions  was  to  illicit  responses  about  the 
overall  debriefing  process  and  allow  the  respondents  to  make 
any  comments  they  want  to  about  the  debriefing  process  in  an 
attempt  to  get  responses  not  covered  in  the  first  type  of 
questions.  These  questions  focus  on  the  strengths  and 
weaknesses  of  the  debriefing  process  and  how  the  process  can 
be  improved.  These  comments  were  used  in  an  attempt  to 
capitalize  on  the  strengths  of  the  process  and  find  ways  to 
overcome  the  weaknesses  identified. 

D .   INTERVIEWS 

Interviews  were  conducted  with  both  defense  contractor 
representatives  and  AMC  contracting  officials.   The  intent  of 
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the  interviews  was  to  follow  up  on  recurrent  answers  or 
comments  from  the  surveys  and  to  identify  other  areas  of  the 
debriefing  process  that  could  be  improved.  Interviews  of 
defense  contractors  were  conducted  within  the  broad  cross- 
section  of  companies  that  contract  with  AMC.  This  includes 
large  and  small  businesses,  hardware  producers,  and  a  services 
company.  Interviews  were  conducted  first  with  the  defense 
contractors.  This  was  done  so  that  information  on  areas  that 
needed  improvement  and  their  possible  solutions  could  be 
collected  and  then  presented  to  AMC  contracting  officials. 
These  officials  could  then  comment  on  the  identified  areas  and 
feasibility  of  the  possible  solutions.  Additionally,  they 
could  address  any  areas  they  wanted  to  and  present  possible 
solutions  to  any  weak  areas  of  the  debriefing  process. 
Interviews  were  conducted  with  contracting  officers  from  three 
of  the  five  MSC's  that  were  surveyed. 

All  interviews  are  recorded  as  anonymous  entries  in  the 
thesis.  This  was  done  to  ensure  more  complete  and  honest 
comments  would  be  obtained  by  not  identifying  the  source  of 
the  comments.  This  avoids  the  concern  on  the  part  of 
interview  participants  that  their  comments  could  be  viewed 
negatively  by  any  others  in  the  AMC  procurement  system. 
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E.   SUMMARY 

This  chapter  outlined  the  methods  of  research  used  in  the 
thesis.  They  include  a  literature  search,  survey 
questionnaires,  and  interviews.  It  identified  the  purpose  for 
each  method.  The  target  audience  and  survey  design  for  the 
survey  questionnaires  are  fully  explained.  Lastly,  details  of 
how  the  interviews  were  done  was  covered.  Chapter  IV  will 
present  and  analyze  the  data  obtained  from  the  surveys  and 
interviews. 


43 


IV.   DATA  PRESENTATION  AND  ANALYSIS 

A.  GENERAL 

This  chapter  contains  the  data  presentation  and  analysis. 
It  contains  two  major  sections.  The  first  is  data 
presentation  and  analysis  for  the  survey  questionnaires  sent 
out  and  the  second  covers  interview  comments  and  analysis  of 
these  comments. 

B.  SURVEY  QUESTIONNAIRES 

The  questions  are  discussed  as  they  appear  on  each  of  the 
surveys.  A  discussion  of  the  answers  and  an  analysis  of  the 
answers  follow.  The  questions  from  the  two  surveys  that  are 
similar  are  presented  and  analyzed  together. 

1.  Question  1  The  intent  of  this  question  is  to 
determine  what  percent  of  unsuccessful  offerors  request  a 
debriefing. 

a.   Question  to  unsuccessful  offerors 

When  not  awarded  a  contract  I  request  a  debriefing 

a.  more  than  2/3 's  of  the  time 

b.  between  1/3  and  2/3 's  of  the  time 

c.  less  than  1/3  of  the  time 
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1.   Discussion 

Forty  two  (42)  contracting  officials  answered 

this  question.  Sixty  (60)  percent  responded  that  they  request 

a  debriefing  more  than  2/3' s  of  the  time,  24%  answered  that 

they  request  a  debriefing  between  1/3  and  2/3 's  of  the  time 

and  16%  reported  that  they  wanted  a  debriefing  less  than  1/3 

of  the  time.   Many  comments  were  included  with  the  multiple 

choice  answers.   Seventeen  (17)  percent  of  the  respondents 

reported  that  they  always  request  a  debriefing,  10%  commented 

that  the  value  of  the  contract  influences  their  decision  to 

request  a  debriefing.   The  larger  the  contract  the  more  apt 

they  are  to  request  a  debriefing.    Finally  5%  of  the 

respondents  said  that  they  request  a  debriefing  only  when  they 

don't  understand  why  they  didn't  win  the  contract  award. 

b.   Question  to  AMC 

How   many   unsuccessful   offerors   ask   for   a 
debriefing? 

a.  less  than  1/3 

b.  between  1/3  and  2/3 's 

c.  more  than  2/3 's 

1.   Discussion 

Thirty  two  (32)  respondents  answered  this 
question.  Seventy  five  (75)  percent  reported  that  more  than 
2/3 's  of  unsuccessful  offerors  ask  for  a  debriefing,  12% 
commented  that  between  1/3  and  2/3 's  of  unsuccessful  offerors 
ask  for  a  debriefing  and  12%  stated  that  less  than  1/3  of 
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unsuccessful  offerors  ask  for  a  debriefing.  Several  comments 
were  added  to  the  answers.  Sixteen  (16)  percent  of  the  AMC 
officials  wrote  that  usually  all  unsuccessful  offerors  ask  for 
a  debriefing.  Nineteen  (19)  percent  reported  that  for  large 
solicitations  normally  all  unsuccessful  offerors  ask  for  a 
debriefing. 

c.   Analysis 

The  feedback  from  the  two  surveys  is  very  similar. 
Both  surveys  report  that  a  high  percent  of  unsuccessful 
offerors  request  debrief ings.  Additionally,  both  surveys  have 
comments  that  support  the  fact  that  as  a  solicitation 
increases  in  value  the  debriefing  for  an  unsuccessful  offeror 
becomes  more  important.  The  results  of  this  question  confirm 
that  debriefings  are  an  important  source  of  information  for 
unsuccessful  offerors. 
2 .   Question  2 

The  intent  of  this  question  is  to  determine  if  AMC 
officials  give  debriefings  when  they  are  requested. 

a.   Question  to  unsuccessful  offeror 

The  Army  gives  a  debriefing  every  time  I  request 


one. 


a.  yes 

b.  no 
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1.   Discussion 

Forty  two  (42)  unsuccessful  offerors  answered 
this  question.  Eighty  eight  (88)  percent  reported  that  they 
always  get  a  debriefing  and  12%  answered  that  they  don't 
always  get  a  debriefing  when  they  request  one.  Four  comments 
were  received.  One  contractor  that  answered  no  stated  that  it 
depends  on  the  commodity  command  and  another  reported  that  he 
received  one  about  95%  of  the  time  he  asked  for  one.  Two 
comments  were  received  from  contractors  who  answered  yes.  One 
stated  that  AMC  officials  may  try  to  do  the  debriefing  on  the 
telephone  and  the  other  said  that  some  debriefings  he  receives 
are  nearly  worthless. 

b.   Question  to  AMC 

I  give  a  debriefing  every  time  one  is  requested. 

a.  yes 

b.  no 

1.   Discussion 

Thirty  two  (32)  answers  were  received.  All  32 
AMC  contracting  officials  reported  that  they  always  gave  a 
debriefing  when  one  was  requested.  Three  respondents 
commented  that  they  feel  a  refusal  to  give  a  debriefing  may 
lead  to  a  protest.  One  official  commented  that  this  is 
required  by  the  FAR.  One  stated  that  they  know  of  no  reason 
not  to  give  a  debriefing  if  one  is  requested,  and  another  said 
that  formal  face  to  face  debriefings  are  rare. 
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c.   Analysis 

There  is  a  difference  in  answers  between  the  two 
participants  here.  Twelve  (12)  percent  of  the  contractors 
stated  that  they  didn't  always  receive  a  debriefing  when  they 
requested  one  and  all  the  AMC  contracting  officials  reported 
that  they  always  gave  a  debriefing  when  one  was  requested. 
The  true  answer  is  probably  somewhere  in  between.  The  AMC 
contracting  officers  may  not  want  to  admit  to  not  giving  a 
debriefing  or  may  have  misunderstood  a  request  for  one  as 
something  else.  It  is  also  possible  that  some  requests  never 
get  to  the  contracting  officer  and  the  contractor  didn't 
follow  up  on  the  request  to  ensure  the  contracting  officer  was 
aware  that  a  debriefing  was  requested.  Additionally,  there 
may  be  some  mistake  on  the  contractors  part  as  to  when  a 
debriefing  is  authorized  and  they  may  request  debrief ings  when 
one  is  not  allowed.  They  would  then  not  get  one  but  think 
they  were  entitled  to  one. 
3.   Question  3 

The  intent  of  this  question  is  to  determine  who 
attends  debrief ings. 

a.   Question  to  unsuccessful  offeror 

The  following  personnel  (by  job  title)  from  my 
company  attend  the  debriefing  . 
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1.   Discussion 

Many  answers  were  given  for  this  question. 
All  answers  included  more  than  one  person.  The  most 
frequently  named  people  were  the  director  of  contracting,  the 
business  development/program  manager  and  the  director  of 
engineering.  Results  of  this  question  are  shown  in  Table  1. 
TABLE  1:  DEFENSE  CONTRACTOR  PERSONNEL  THAT  ATTEND  DEBRIEFINGS 


Job  Title 

Times  Mentioned 

Director  of  Contracting 

28 

Business  Dev/Program  Manager 

27 

Director  of  Engineering 

15 

Proposal  Manager 

13 

Marketing  Manager 

12 

Vice  President 

10 

President 

4 

Vice  President-  Engineering 

3 

Pricing  Manager 

3 

Director  of  Operations 

2 

Department  Manager 

2 

Contract  Administrator 

2 

Legal  Counsel 

2 

Director  of  Technology 

1 

SOURCE:  DEVELOPED  BY  RESEARCHER 


b.   Question  to  AMC 

The  following  personnel  (by  job  title)  from  my 
organization  attend  the  debriefing  . 
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1.   Discussion 

Many  people  were  mentioned  in  the  answers 
received  for  this  question.  The  most  common  responses  were 
the  contracting  officer,  listed  by  all  respondents,  the 
contract  specialist,  and  SSEB  chairman  and  factor  chairs. 
Four  of  the  responses  listed  the  contracting  officer  as  the 
sole  person  who  attends  from  their  organization.  Results  of 
this  question  are  annotated  in  Table  2. 
TABLE  2:  AMC  PERSONNEL  THAT  ATTEND  DEBRIEFINGS 


Job  Title 

Times  Mentioned 

Contracting  Officer 

32 

Contract  Specialist 

19 

Legal  Council 

16 

Factor  Chairs 

14 

Technical  Specialist 

11 

SSEB  Chairman 

9 

Project  Officer 

6 

Price  Analyst 

5 

Price  Analyst 

5 

SSEB  Deputy 

4 

PRAG  Chairman 

1 

SOURCE:  DEVELOPED  BY  RESEARCHER 

c.   Analysis 

The  results  of  this  question  show  that  the 
debriefing  is  not  a  one  man  show  for  either  side.  Rather  many 
people  attend  debriefings  and  in  many  cases  the  debriefing  is 
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seen  as  important  enough  that  SSEB  chairman  and  vice 
presidents  of  companies  attend.  The  job  titles  of  employees 
that  attend  for  unsuccessful  offerors  indicates  that  the  firms 
view  the  debriefing  as  an  integral  part  of  their  effort  to 
understand  why  they  lost  the  contract  award  and  identify  what 
steps  they  can  take  to  be  more  competitive  in  future 
solicitations . 

4.   Question  4 

This  question  was  designed  to  determine  if  AMC  limits 
the  number  of  people  who  may  attend  debrief ings. 

a.   Question  to  unsuccessful  offeror 

The  Army  limits  the  number  of  employees  that  may 
attend  the  debriefing  from  my  company. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Forty  one  (41)  offerors  answered  this 
question.  Ten  (10)  percent  responded  that  the  Army  always 
limits  the  number  of  employees,  59%  replied  that  they 
sometimes  limit  the  number,  and  30%  stated  that  the  Army  never 
limits  the  number  of  employees  that  can  attend  debrief ings. 
Three  comments  were  received  on  this  question.  Two  stated 
that  employee  numbers  were  only  limited  if  the  conference  room 
wasn't  big  enough  to  handle  more  people.  One  contractor 
stated  that  this  was  infrequently  a  problem. 
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b.   Question  to  AMC 

I  limit  the  number  of  employees  that  unsuccessful 
offerors  may  send  to  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  one  (31)  AMC  respondents  answered  this 
question.  Three  (3)  percent  stated  that  they  always  limit  the 
number  of  employees,  39%  said  they  sometimes  limit  the  number, 
and  58%  stated  that  they  never  limit  the  number  of  employees 
that  may  attend  a  debriefing.  There  were  several  comments 
made  to  this  question.  Three  Contracting  officials  wrote  that 
sometimes  the  conference  room  isn't  large  enough,  especially 
on  large  procurements  were  many  participants  from  both  sides 
attend.  Six  people  stated  that  the  number  of  employees  the 
contractor  sends  has  never  been  a  problem.  One  stated  that 
the  available  space  in  the  conference  room  dictates  the  number 
that  may  attend.  One  respondent  wrote  that  limiting  spaces 
insures  that  only  those  people  necessary  to  understand  why 
their  company  lost  the  contract  award  attend  the  debriefing. 
Two  comments  referred  to  lawyers  attending.  One  person  stated 
that  he  didn't  limit  the  number  of  people  attending  but 
requested  a  list  of  people  by  job  title  who  would  attend  the 
debriefing.  This  is  done  to  determine  if  a  lawyer  is  going  to 
attend   the   debriefing,   which  would   indicate   a   hostile 
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contractor.   Another  person  said  that  limiting  the  number  of 
people  attending  depends  on  if  you  know  whether  or  not  the 
debriefing  will  be  hostile  as  you  don't  want  to  brief  lawyers 
who  are  looking  for  free  discovery. 
c.   Analysis 

Both  the  unsuccessful  offerors  and  AMC  officials 
report  that  there  are  times  when  the  number  of  employees  from 
the  unsuccessful  firm  that  may  attend  a  debriefing  are 
limited.  The  unsuccessful  offerors  state  that  it  happens  at 
a  much  higher  rate  than  AMC  officials  claim  it  happens.  Only 
3  0%  of  the  contractors  said  they  were  never  limited  to  the 
number  of  employees  that  could  attend  a  debriefing  while  58% 
of  the  AMC  officials  stated  they  never  limit  the  number  of 
employees.  The  true  answer  is  probably  somewhere  in  between. 
The  discrepancy  between  these  percentages  may  be  partially 
explained  by  a  misunderstanding  on  the  part  of  contractor 
personnel  of  who  limits  the  number  of  people  attending.  In 
some  cases  it  may  be  their  own  company  limiting  the  numbers  in 
an  attempt  to  keep  travel  costs  down.  Additionally,  some  AMC 
officials  may  not  have  provided  an  accurate  answer  in  an 
attempt  to  downplay  any  effect  from  limiting  the  number  of 
employees  that  may  attend  a  debriefing. 
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5.    Question  5    The  intent  of  this  question  is  to 

determine  if  AMC  officials  are  limiting,  by  job  title,  the 

employees  that  may  attend  debrief ings  from  unsuccessful  firms. 

a.   Question  to  unsuccessful  offerors 

The  Army  limits  the  number  of  employees,  by  job 
position  that  may  attend  the  debriefing  from  my  company. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  nine  (39)  answers  were  received  for 

this  question.   Three  (3)  percent  responded  that  AMC  always 

limits  the  number,  by  job  position,  16%  stated  that  they  are 

sometimes  limited,  and  82%  said  that  they  are  never  limited  by 

job  position.   Two  comments  were  included  with  the  answers. 

One  stated  that  they  were  sometimes  limited  to  sending  the 

senior  staff,  and  the  other  said  that  they  were  not  limited 

but  AMC  wants  a  list  beforehand  of  who  they  are  going  to  send. 

b.   Question  to  AMC 

I  limit  employees,  by  job  title,  that  unsuccessful 
offerors  may  send  to  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  (30)  responses  were  received  for  this 
question.   Seven  (7)  percent  stated  that  they  sometimes  limit 
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employees,  by  job  position,  from  attending  a  debriefing  and 
93%  said  that  they  never  limit  the  employees,  by  job  title. 
Three  respondents  commented  that  the  unsuccessful  offeror  may 
send  any  employees  they  want  to.  One  stated  that  he  never 
limits  the  employees,  by  job  position,  but  does  ask  for  a  list 
of  who  will  attend  ahead  of  time.  Another  commented  that  he 
gets  a  list  ahead  of  time  and  uses  it  as  a  going  in  position 
in  order  to  prepare  and  organize  a  positive  and  professional 
debriefing. 

c.   Analysis 

There  is  a  disparity  between  the  percentage  of 
responses  for  the  two  sides  that  report  a  limit,  by  job 
position,  of  who  may  attend  a  debriefing  from  the  unsuccessful 
firm.  Only  7%  of  AMC  personnel  report  ever  limiting  employees 
by  job  position  while  18%  of  unsuccessful  firms  report  that 
this  has  happened.  Possible  reasons  for  this  difference  could 
be  a  misunderstanding  on  the  contractor's  part  of  who  places 
this  limit.  In  some  cases  it  may  be  the  company  and  not  AMC. 
Additionally  if  this  ever  happened  it  would  be  the  contractor 
who  recalled  it  happening  first  and  it  may  be  of  minor 
significance  to  AMC  personnel  and  thus  forgotten. 

6.   Question  6   This  question  was  designed  to  determine 
how  long  after  a  request  debrief ings  are  held. 
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a.   Question  to  unsuccessful  offerors 

Debriefings   are   normally   held   within   

calendar  days  of  my  request. 

a.  0-10 

b.  11-20 

c.  more  than  20 

1.   Discussion 

Answers  were  received  from  42  contractors. 

21%  responded  that  debriefings  were  held  within  10  days. 

Forty  eight  (48)  percent  said  between  11-20  days  and  the 

remaining  31%  stated  more  than  20  days.   Only  one  comment  was 

reported.   The  contractor  wrote  that  the  time  varied  but  was 

usually  close  to  or  in  excess  of  30  days. 

b.   Question  to  AMC 

Debriefings  are  held  within  calendar  days  of 

request . 

a.  0-10 

b.  11-20 

c.  more  than  20 

1.   Discussion 

Thirty  one  (31)  AMC  officials  responded  to 
this  question.  Eighty  four  (84)  percent  stated  that 
debriefings  were  done  within  10  days  of  request.  The  other 
16%  said  it  was  done  between  11-20  days.  Many  comments  were 
written  in  response  to  this  question.  One  respondent  wrote 
that  it  is  best  to  schedule  debriefings  as  soon  as  possible  so 
that  SSEB  members  are  available  to  assist.   One  said  that  he 
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is  always  prepared  to  debrief  the  day  after  notification  of 
unsuccessful  offerors.  A  delay  gives  the  appearance  that  you 
had  to  stop  and  document  why  the  offeror  lost.  Another 
respondent  said  that  you  should  allow  sufficient  time  to 
prepare  adequately  and  still  have  the  decision  process  fresh 
in  mind.  One  contracting  officer  wrote  that  his  debriefs  are 
normally  built  into  the  acquisition  cycle  schedule  and  pre- 
scheduled  for  the  first  3  days  after  contract  award.  Another 
AMC  official  said  that  he  did  it  within  10  days  of  request,  as 
early  debrief ings  help  offset  possible  protests.  Three 
respondents  stated  that  the  time  depends  on  the  number  of 
offerors  and  when  both  parties  can  get  together. 
c.   Analysis 

There  is  a  significant  difference  of  opinion 
between  the  two  parties  as  to  how  long  after  the  request  the 
debrief  ings  are  held.  Eighty  four  (84)  percent  of  AMC 
personnel  said  debrief ings  were  held  within  10  days  of  request 
while  only  21%  of  the  unsuccessful  offerors  reported  that  they 
were  done  within  this  same  time  frame.  One  reason  for  this 
difference  could  result  from  the  requirement  for  the 
debriefing  request  to  be  written.  If  these  written  requests 
are  mailed  contractors  may  consider  the  clock  to  start  on 
their  request  when  they  drop  their  request  in  the  mail.  AMC 
officials  would  most  likely  start  the  clock  when  they  receive 
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the  request .  This  reason  alone  would  not  solve  the  vast 
difference  in  opinion  between  the  two  parties  on  this  issue. 
When  a  debriefing  is  done  can  have  an  effect  on  if  a  protest 
is  filed  or  not.  This  in  turn  can  have  an  effect  on  any- 
subsequent  debriefing  given.  Contractors  have  10  days  after 
contract  award  is  announced  to  file  a  protest  which  will  stop 
work  on  the  contract.  The  survey  shows  if  a  debriefing  is  not 
given  within  these  10  days  a  contractor  may  file  a  protest 
because  he  may  suspect  some  type  of  impropriety  in  the  source 
selection  decision  process.  A  debriefing  done  within  the  10 
day  window  may  show  the  contractor  there  was  no  impropriety  in 
the  decision  process  and  no  protest  is  necessary. 
Additionally,  any  debriefings  given  to  unsuccessful  offerors 
that  have  already  protested  the  contract  award  may  not  be  very 
informative  because  the  contracting  officer  is  on  the 
defensive  and  is  not  looking  to  give  the  unsuccessful  offeror 
information  that  could  be  used  in  the  protest. 

7.  Question  7  The  intent  of  this  question  is  to 
determine  when  the  unsuccessful  offerors  and  AMC  officials 
think  the  debriefing  should  be  done. 

a.   Question  to  Unsuccessful  offeror 

Debriefings   should  be   conducted  within  

calendar  days  after  contract  award. 
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1.   Discussion 

Thirty  eight  (38)  contractors  answered  this 

question.   Answers  received  ranged  from  5  to  30  days.   Fifty 

(50)  percent  of  those  that  responded  said  that  the  debriefing 

should  be  done  within  10  days.   The  average  answer  to  this 

question  is    14.4  days.    Three  different  comments  were 

included  with  the  answers.   Five  contractors  responded  that 

the  debriefing  must  be  done  within  the  limits  of  the  time  to 

protest.   Two  people  stated  that  the  sooner  the  better  and 

that  sooner  may  help  head  off  unnecessary  protests.   Lastly, 

one  official  stated  that  he  recognized  the  difficulty  of 

getting  all  the  AMC  people  together  but  earlier  feedback  is 

better. 

b.   Question  to  AMC 

Debriefings   should  be   conducted  within  

calendar  days  after  contract  award. 

1.   Discussion 

Twenty  eight  (28)  AMC  officials  answered  this 
question.  The  answers  ranged  from  4  to  3  0  days  with  the 
average  being  14.5  days.  Fifty  three  (53)  percent  said  that 
debriefings  should  be  done  within  10  calendar  days  after 
contract  award.  Several  respondents  provided  comments  with 
their  answer.  Two  stated  that  the  FAR  says  to  conduct  the 
debriefings  as  soon  as  possible.  One  stated  that  the  SSEB 
goes  to  the  four  winds  after  award  so  it  is  best  to  do  it  as 
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soon  as  possible.  One  official  commented  that  the  10  day- 
protest  cut  off  is  a  factor,  however  SSEB  support  is  the  final 
determining  factor.  Another  comment  was  that  many  protests 
are  filed  after  a  debriefing  so  the  debriefing  should  be  done 
quickly.  Two  officials  said  to  let  the  offeror  decide  when 
the  debriefing  should  be  done  and  honor  their  request. 

c.   Analysis 

The  responses  from  both  the  contractors  and  AMC 
officials  indicate  that  debriefings  should  be  done  in  a  timely 
manner.  The  average  response  for  both  groups  show  that 
debriefings  should  be  done  within  14  days  of  contract  award. 
8.  Question  8  The  intent  of  this  question  is  to 
determine  who  normally  conducts  the  debriefing. 

a.   Question  to  unsuccessful  offeror 
normally  debriefs  me. 

a.  the  contracting  officer 

b.  the  source  selection  evaluation  board  chairman 

c.  a  combination  of  both  a  &  b 

d.  other  

1.   Discussion 

Forty  two  (42)  defense  contractors  answered 
this  question.  Twenty  one  (21)  percent  responded  that  the 
contracting  officer  debriefs  them,  74%  said  that  a  combination 
of  the  contracting  officer  and  SSEB  chairman  debrief  them  and 
5%  stated  that  someone  else  debriefs  them.  This  other  person 
was  the  program  manager.   Several  comments  were  received  that 
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added  names  of  others  who  participate.  Lawyers  were  mentioned 
3  times,  technical  evaluators  were  listed  on  4  surveys,  cost 
analysts  were  mentioned  2  times  and  a  contract  specialist  was 
listed  once.  One  additional  comment  was  that  when  the 
contract  officer  is  the  only  person  to  debrief  the  debriefing 
tends  to  be  dry  and  not  very  informative. 
b.   Question  to  AMC 

Debrief ings  are  conducted  by  (give  job  title) 


1.   Discussion 

Five  different  answers  were  received  for  this 
question.  Thirty  eight  (38)  percent  of  the  respondents 
indicated  that  the  contracting  officer  conducts  the 
debriefing.  Nineteen  (19)  percent  said  the  contracting 
officer  and  the  contract  specialist  conduct  the  debrief ings, 
and  19%  stated  the  contracting  officer  and  technical  experts 
do  it.  Twenty  five  (25)  percent  of  the  people  said  the 
contracting  officer  and  the  SSEB  conduct  debrief ings. 
c.   Analysis 

Both  the  contractors  and  AMC  officials  answers  to 
this  question  reveal  that  a  contracting  officer  participates 
in  almost  all  debrief ings.  They  are  generally  joined  by  such 
personnel  as  the  SSEB  chairman,  SSEB  members,  technical 
representatives,  and  contract  specialist.   It  is  clear  though 
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that  the  contracting  officer  is  the  key  player  for  AMC  in  the 
debriefing  process.  This  coincides  with  what  the  AFARS 
recommends . 

9.   Question  9   This  question  was  designed  to  determine 
where  debrief ings  are  held. 

a.   Question  to  unsuccessful  offerors 

Debriefings  are  normally  held  at  . 

a.  at  my  location 

b.  at  the  Army's  location 

c.  other  location 

d.  by  telephone 

1.   Discussion 

Forty  two  (42)  contractors  answered  this 
question  with  2  contractors  giving  2  answers.  Eighty  six  (86) 
percent  of  the  responses  indicated  that  the  debriefings  are 
held  at  the  Army's  location.  Fourteen  (14)  percent  of  the 
answers  reported  that  they  are  done  by  telephone.  Two 
contractors  listed  both  of  these  answers.  Two  different 
comments  accompanied  these  answers.  Three  contractors  who 
answered  that  debriefings  are  normally  held  at  the  Army's 
location  stated  that  sometimes  telephone  debriefings  are  done. 
One  of  these  officials  stated  this  is  usually  a  conference 
call.  Another  of  these  officials  said  that  the  telephone 
debriefings  are  frequently  done  when  the  AMC  command  is  out  of 
town. 


62 


b.   Question  to  AMC 

Debriefings  are  normally  held 


a.  at  my  location 

b.  at  the  unsuccessful  offeror's  location 

c.  other  location 

d.  by  telephone 

1.   Discussion 

Thirty  four  (34)  answers  were  received. 
Eighty  eight  (88)  percent  of  the  answers  indicated  that 
debriefings  are  normally  held  at  the  AMC  location  while  12% 
reported  they  are  done  by  telephone.  Two  contracting  officers 
stated  both  of  these  answers.  Eight  AMC  personnel  commented 
that  debriefings  are  sometimes  done  by  phone.  Four  of  these 
people  added  that  this  was  especially  so  for  small 
procurements.  Two  officials  stated  that  debriefings  should 
never  be  done  at  the  unsuccessful  offeror's  location, 
c.   Analysis 

The  answers  to  this  question  are  almost  identical 
for  each  survey.  The  results  indicate  that  most  debriefings 
are  done  in  person  at  the  AMC  location.  A  small  percent  of 
the  debriefings  are  done  by  phone,  especially  when  the 
procurement  is  small  or  when  the  AMC  location  is  out  of  town. 
Not  one  respondent  reported  that  any  debriefing  was  ever  done 
at  the  offeror's  location.  All  of  this  indicates  that  there 
is  an  unwritten  practice  to  not  conduct  debriefings  at  the 
offeror's  location.    If  the  offeror  wants  the  service  in 


63 


person  he  must  travel  to  get  it.  The  reason  contractors  will 
request  debrief ings  by  phone  for  smaller  procurements  is  that 
they  are  not  willing  to  pay  for  travel  costs.  The  costs  would 
outweigh  the  benefits  in  these  cases.  Instead  they  will  get 
as  much  information  as  they  can  by  telephone. 
10.   Question  10 

The  intent  of  this  question  is  to  determine  if 
questions  are  allowed  and  answered  in  debrief ings. 

a.   Question  to  unsuccessful  offeror 
The  Army  . 

a.  doesn't  allow  questions 

b.  allows  and  answers  all  questions 

c.  allows  and  answers  some  questions 

1.   Discussion 

Forty  two  (42)  answers  were  received  for  this 
question.  Seven  (7)  percent  of  the  respondents  said  that  the 
Army  doesn't  allow  questions.  Two  (2)  percent  stated  that 
questions  are  allowed  and  they  are  all  answered.  Ninety  one 
(91)  percent  indicated  that  the  Army  allows  and  answers  some 
questions.  Four  comments  reported  that  the  answers  were  not 
detailed  enough.  Two  contractors  commented  that  they  were 
limited  to  asking  questions  about  their  proposal  only.  One 
offeror  stated  that  allowing  and  answering  questions  really 
depends  on  who  the  contracting  officer  is. 
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b.   Question  to  AMC 

I  . 

a.  don't  allow  questions 

b.  allow  and  answer  all  questions 

c.  allow  and  answer  some  questions 

1.   Discussion 

All  32  contracting  officers  answered  this 
question.  Thirty  eight  (38)  percent  stated  that  they  allow 
and  answer  all  questions  and  62%  said  that  they  allow  and 
answer  some  questions.  Several  comments  were  received.  Eight 
respondents  said  that  they  don't  allow  questions  about  another 
offeror's  proposal.  Three  stated  that  they  answer  questions 
to  the  extent  the  regulations  allow.  Two  officials  said  that 
they  may  answer  some  questions  in  writing  after  the 
debriefing.  One  contracting  officer  stated  that  he  can't 
answer  some  questions  and  some  aren't  appropriate.  Another 
said  that  all  questions  and  answers  must  be  in  writing  and 
that  the  openness  of  the  debrief  depends  on  the  likelihood  of 
a  protest.  Lastly,  one  official  wrote  that  he  uses  a  caucus 
between  questions  and  answers. 
c.   Analysis 

There  is  a  large  difference  between  the  two 
surveys  with  the  answer  that  the  Army  allows  and  answers  all 
questions.  Thirty  eight  (38)  percent  of  AMC  personnel  gave 
this  answer  versus   only  2%  of   the   contractors.    Some 
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interpretations  of  the  possible  survey  answers  may  have  been 

done  to  cause  this.  It  doesn't  seem  likely  that  AMC  officials 

would  answer  all  questions  asked  by  contractors,  especially  in 

the  contractor's  quest  to  get  information  on  the  winner's 

proposal .   Consequently  a  more  accurate  answer  for  AMC  is 

probably  much  lower.   Several  contractors  reported  that  they 

are  not  allowed  to  ask  questions  during  a  debrief.   This 

doesn't  allow  the  contractor  to  get  the  information  he  needs 

to  understand  the  evaluation  of  his  proposal. 

11.   Question  11 

This  question  was  designed  to  determine  why  some 

questions  asked  by  unsuccessful  offerors  are  not  answered. 

a.   Question  to  unsuccessful  offeror 

If  the  Army  doesn't  answer  some  of  my  questions  it 
is  because 

a.  they  can't  answer  them  due  to  regulations  or 
confidential/proprietary  business  information 

b.  they  won't  answer  them  due  to  their  concern  for  a 
protest 

c.  other  

d.  not  applicable,  they  answer  all  my  questions 

1.   Discussion 

A  total  of  56  answers  were  recorded  for  this 
question.  Although  the  question  was  designed  to  get  only  one 
answer  from  each  respondent,  15  of  the  41  officials  who 
answered  this  question  gave  two  responses.  These  15  answered 
that  the  Army  doesn't  answer  some  questions  due  to  both 
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regulations  and  confidential/proprietary  business  information, 

and  their  concern  for  a  protest.   Fifty  nine  (59)  percent  of 

the    answers    indicated    that    regulations    or 

confidential/proprietary  business  information  is  the  reason 

the  Army  doesn't  answer  all  questions.    Thirty  six  (36) 

percent  of  the  answers  pointed  to  a  concern  for  protests  as 

the  reason.   Five  (5)  percent  of  the  respondents  listed  other 

reasons  which  were  that  the  question  is  not  relevant  or  is 

source  selection  sensitive  or  classified,  they  don't  know  the 

answer  to  the  questions  they  don't  answer,  and  a  hidden  agenda 

prevents  them  from  answering  some  questions.   One  comment  was 

recorded.   It  stated  that  AMC  personnel  tend  to  error  on  the 

conservative  side  when  determining  when  to  answer  a  question. 

b.   Question  to  AMC 

If  I  don't  answer  some  of  the  unsuccessful 
offerors  questions  it  is  because 

a.  I  can't  answer  them  due  to  regulations  or 
confidential/proprietary  business  information 

b.  I  won't  answer  them  due  to  a  possible  protest 

c.  other  

d.  not  applicable,  I  answer  all  questions 

1.   Discussion 

Thirty  five  answers  were  received  for  this 

question.  Three  AMC  officials  gave  two  answers.  Eighty  three 

(83)  percent  of  the  answers  indicated  that  when  questions 

aren't    answered    it    is    due    to    regulations,    or 

confidential/proprietary  business  information.    Nine   (9) 
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percent  of  the  answers  stated  some  questions  aren't  answered 
for  other  reasons.  These  reasons  are  when  the  offeror  is 
seeking  opinion  and  not  facts,  and  if  information  is  not 
available  and  then  an  attempt  will  be  made  to  provide  it 
later.  Two  surveys  gave  this  latter  reason  as  their  answer. 
Contracting  officers  submitted  several  comments  to  this 
question.  Two  stated  that  they  don't  worry  about  protests  and 
answer  as  many  questions  as  possible.  One  of  these  continued 
that  the  Freedom  of  Information  Act  allows  most  information  to 
be  disclosed  so  you  might  as  well  give  it  to  them  up  front. 
The  other  added  that  the  Government  Accounting  Office  will  let 
a  protestor  see  everything  during  discovery  so  he  doesn't  hide 
anything.  Another  official  stated  that  when  there  is  the 
potential  for  a  protest  it  makes  his  answers  rather  anemic. 
One  official  stated  that  he  doesn't  answer  questions  that 
compare  proposals,  disclose  relative  rankings  or  discuss 
weights . 

c.   Analysis 

Both  sides  agree  that  many  questions  are  not 
answered  due  to  regulations  or  confidential/proprietary 
business  information.  However  a  higher  percentage  of 
contracting  officers  than  unsuccessful  offerors  stated  this  as 
the  reason  some  questions  aren't  answered.  This  is  most 
likely  a  result  of  how  the  FAR  and  AFARS  are  interpreted.   It 
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is  the  contracting  officer's  responsibility  to  operate  within 
the  regulations,  so  they  are  much  more  likely  to  take  a  more 
conservative  view  of  what  these  regulations  mean.  The  two 
survey  groups  disagree  on  the  issue  of  not  answering  questions 
due  to  a  fear  of  being  protested.  Not  one  contracting  officer 
answered  that  fear  of  a  protest  prevents  them  from  answering 
a  question  while  46%  of  the  contractors  stated  that  this  is 
the  reason  some  questions  aren't  answered.  This  leads  to  the 
conclusion  that  contracting  officers  feel  they  are  answering 
all  they  can  legally  answer  while  many  contractors  believe 
questions  that  could  be  answered  are  not  being  answered. 

12.  Question  12  This  question  was  designed  to  determine 
if  contracting  officers  pass  along  all  the  information  that 
unsuccessful  offerors  are  entitled  to  according  to  the 
regulations . 

a.   Question  to  unsuccessful  offerors 
Debriefings  normally  consist  of 

a.   limited  information,  because 


b.  as  much  information  as  the  Army  can  give  in 
accordance  with  the  regulations  and  confidential/proprietary 
business  information 

1.   Discussion 

All  42  respondents  answered  this  question. 
48%  stated  that  the  debriefings  consisted  of  limited 
information.  Fifty  five  (55)  percent  of  these  people  said  the 
reason  was  due  to  the  fear  of  a  protest.  Two  said  it  was  due 
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to  proprietary  data  rights  and  three  others  said  it  was  due  to 
restrictive  regulations.  Three  contractors  stated  that  it  was 
because  of  an  attempt  to  cover  up  improper  procurement 
practices.  The  last  reason  given  was  that  the  contracting 
officers  don't  want  to  compare  the  offeror  with  the  winner. 
The  other  52%  of  the  respondents  answered  that  the  Army  gives 
out  as  much  information  as  it  can  under  the  current 
regulations  and  confidential/proprietary  business  information 
stipulations.  Two  comments  to  the  question  were  given.  One 
was  that  certain  contracting  officers  give  very  detailed 
debrief ings  while  others  give  generic  debrief ings.  The  other 
stated  that  it  depends  on  what  MSC  you're  dealing  with  as  some 
provide  more  information  than  others. 

b.   Question  to  AMC 

Debriefings  normally  consist  of 

a.   limited  information,  because  


b.  as  much  information  as  I  can  give  in  accordance 
with  the  regulations  and  confidential/proprietary  business 
information 

1.   Discussion 

Thirty  one  contracting  officers  answered  this 
question  and  all  stated  that  they  give  out  as  much  information 
as  they  can  in  accordance  with  the  regulations  and 
confidential/proprietary  business  data.  Many  comments  were 
provided  with  the  answers.  Five  people  stated  that  providing 
all  the  information  you  can  helps  prevent  protests  and  other 
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problems  such  as  congressional  inquiries.  One  contracting 
officer  stated  that  he  only  gives  out  the  minimum  information 
necessary.  Another  said  that  he  gives  only  a  summary 
debriefing  and  provides  additional  information  if  asked.  One 
person  stated  that  he  thinks  AMC  is  extremely  conservative  in 
what  information  it  gives  out.  One  comment  indicated  that 
contractors  want  information  that  compares  their  proposal  to 
the  winner's.  Lastly,  one  official  said  his  command  now  gives 
out  the  same  information  to  the  unsuccessful  offeror  on  his 
proposal  that  is  given  to  the  SSA. 
c.   Analysis 

There  is  a  large  difference  between  what  the  AMC 
officials  and  the  unsuccessful  offerors  believe  as  to  the 
extent  of  the  information  given  out  in  debrief ings.  All  the 
contracting  officers  said  that  they  give  out  what  they  can, 
(although  some  of  their  comments  refute  this)  while  almost  one 
half  of  the  contractors  said  that  the  debriefings  consist  of 
only  limited  information.  While  contracting  officers 
apparently  feel  that  they  give  out  all  the  information  they 
can  there  is  some  evidence  that  certain  commands  and 
contracting  officers  are  willing  to  provide  more  information 
to  unsuccessful  offerors  than  others.  Factors  such  as  fear  of 
protests,  conservative  interpretations  of  the  regulations, 
time  constraints,  and  heavy  work  loads  probably  prevent  some 
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commands  and  contracting  officers  from  providing  all  the  data 
they  could  provide  under  optimum  conditions. 

13.  Question  13  The  intent  of  this  question  is  to 
determine  if  the  threat  of  a  protest  has  any  effect  on  the 
amount  of  information  given  out  in  a  debriefing. 

a.  Question  to  unsuccessful  offerors 

The  Army  would  provide  more  information  if  the 
possible  threat  of  a  protest  did  not  exist. 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 

1.   Discussion 

Forty  two  (42)  answers  were  received  for  this 
question.  Twelve  (12)  percent  of  the  offerors  believe  that 
the  threat  of  a  protest  always  effects  how  much  information  is 
given  out  in  a  debriefing.  Thirty  eight  (38)  percent  believe 
it  sometimes  has  an  effect  and  2%  believe  it  never  has  an 
effect.  Forty  three  (43)  percent  don't  know  if  the  Army  would 
provide  more  information  if  the  threat  of  a  protest  didn't 
exist.  Two  respondents  commented  that  if  this  wasn't  the  case 
then  the  debrief ings  would  be  more  comprehensive. 

b.  Question  to  AMC 

I  would  provide  more  information  if  the  possible 
threat  of  a  protest  did  not  exist. 

a .  always 

b.  sometimes 

c .  never 
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1.   Discussion 

Twenty  nine  (29)  contracting  officers 
responded  to  this  question.  Seven  (7)  percent  said  that  the 
threat  of  a  protest  always  influenced  how  much  information 
they  provide  in  a  debriefing.  Twenty  eight  (28)  percent 
stated  that  a  protest  sometimes  influences  them  and  65%  said 
that  it  never  influences  the  amount  of  information  they 
provide . 

c.   Analysis 

Both  contracting  officers  and  unsuccessful 
offerors  say  that  the  threat  of  a  protest  influences  the 
amount  of  information  that  is  given  out  in  a  debriefing. 
Fully  one  third  of  the  contracting  officers  say  that  it 
effects  them  at  least  sometimes.  Therefore  a  conclusion  can 
be  made  that  the  threat  of  a  protest  influences  the  quality  of 
many  debrief ings.  Contractors  that  are  hostile  or  threaten  a 
protest  before  the  debriefing  is  conducted  are  thus  less 
likely  to  get  the  same  amount  of  information  they  would  get  if 
the  contracting  officer  felt  confident  no  protest  would  be 
filed. 

14.  Question  14  This  question  was  designed  to  determine 
if  debrief  ings  are  conducted  in  accordance  with  the  major 
elements  of  the  RFP. 
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a.   Question  to  unsuccessful  offerors 

The  Army  debriefs  in  accordance  with  the  major 
elements  of  the  RFP. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

All  42  contractors  answered  this  question. 

Fifty  (50)  percent  stated  that  the  Army  always  debriefs  in 

accordance  with  the  RFP.   Forty  four  (44)  percent  said  that 

they  sometimes  do  and  7%  indicated  that  the  Army  never 

debriefs  in  accordance  with  the  RFP.   Two    comments    were 

submitted  with  the  answers.   One  contractor  commented  that 

whether  or  not  the  debriefing  is  done  in  accordance  with  the 

major  elements  of  the  RFP  depends  on  the  command.   Another 

stated  that  only  summary  data  is  provided  for  the  major 

elements . 

b.   Question  to  AMC 

I  debrief  in  accordance  with  the  major  elements  of 
the  RFP. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  (30)  AMC  officials  answered  this 
question.  Ninety  three  (93)  percent  stated  that  they  always 
debrief  in  accordance  with  the  RFP.   Seven  (7)  percent  said 
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that  they  sometimes  do.  One  official  commented  that  he 
usually  limits  the  debriefing  to  only  the  weak  or  deficient 
factors  in  the  proposal .  Another  stated  that  he  gives 
narrative  remarks  for  all  factors  and  sub- factors. 

c.   Analysis 

There  is  a  large  difference  between  the  answers 
given  by  the  unsuccessful  offerors  and  the  AMC  contracting 
officers.  Only  50%  of  the  unsuccessful  offerors  believe  that 
contracting  officers  always  debrief  in  accordance  with  the  RFP 
whereas  93%  of  the  contracting  officers  believe  they  do.  This 
difference  could  be  accounted  for  by  the  amount  of  information 
given  out  for  each  major  element.  If  only  a  small  amount  of 
information  is  given  out  unsuccessful  offerors  may  have 
interpreted  this  as  not  having  been  debriefed  in  accordance 
with  the  major  elements  while  contracting  officers  may  have 
considered  this  being  debriefed  in  accordance  with  the  major 
elements . 

15.  Question  15  This  question  was  designed  to  determine 
if  debriefings  clearly  point  out  weaknesses  in  the 
unsuccessful  offeror's  proposal. 

a.   Question  to  unsuccessful  offerors 

Debriefings  clearly  identify  weaknesses  in  my 


proposal . 


a .  always 

b.  sometimes 

c .  never 
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1.   Discussion 

Forty  two  (42)  responses  were  given  for  this 

question.    Twenty  four   (24)   percent  reported  that  the 

debriefings   always   clearly   identify   weaknesses   in   the 

offeror's  proposal.   Sixty  seven  (67)  percent  said  that  they 

sometimes  did  and  9%  indicated  that  they  never  clearly 

identified  weaknesses  in  their  proposal.   Two  comments  were 

provided.   They  all  indicated  that  improvements  need  to  be 

made  here.  Seven  unsuccessful  offerors  stated  that  there  was 

not  enough  information  given  out  to  determine  exactly  what  the 

weaknesses  were  in  the  proposal .   One  respondent  commented 

that  it  seems  like  just  enough  trivial  weaknesses  are 

identified  to  justify  a  low  cost  award. 

b.   Question  to  AMC 

Debriefings  clearly  identify  weaknesses  in  the 
unsuccessful  offeror's  proposal. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  two  (32)  contracting  officers  answered 
this  question.  Eighty  one  (81)  percent  stated  that  they 
always  clearly  identify  weaknesses  in  unsuccessful  offeror's 
proposals.  Nineteen  (19)  percent  responded  that  they 
sometimes  clearly  identify  weaknesses.  Several  comments  were 
added  to  the  answers.  Two  contracting  officers  commented  that 
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weaknesses  or  deficiencies  not  discussed  prior  to  the 
debriefing  should  not  be  addressed  during  the  debriefing. 
Another  stated  the  same  thing,  but  he  added  that  he  had  an 
experience  with  a  protest  to  GAO  where  GAO  determined  that  any 
weaknesses  that  negatively  affect  the  offeror's  proposal  must 
be  discussed  during  negotiations  or  GAO  may  determine 
meaningful  discussions  were  not  held.  Another  official 
commented  that  weaknesses  are  not  always  available  on  each 
element  due  to  a  lack  of  clear  documentation  in  the  technical 
evaluations.  Two  contracting  officers  commented  that  major 
deficiencies  are  always  addressed  but  weaknesses  are  only 
addressed  if  asked  or  they  had  a  significant  impact  on  the 
source  selection.  The  last  comment  was  that  weaknesses  will 
be  covered  if  they  exist  but  it  is  extremely  difficult  to 
debrief  a  good  proposal  with  no  technical  weaknesses. 
c.   Analysis 

Several  unsuccessful  offerors  and  contracting 
officers  stated  that  debriefings  clearly  identify  weaknesses 
in  many  cases  but  not  all  cases.  This  means  that  not  all  the 
information  that  could  be  given  to  an  unsuccessful  offeror  is 
always  being  given.  Providing  the  offeror  with  his  weaknesses 
allows  the  contractor  to  see  how  his  proposal  stood  up  to  the 
evaluation  criteria  and  more  specifically  where  his  proposal 
did  not  measure  up  to  the  standards  expected  of  it  in 
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accordance  with  the  evaluation  criteria.  Clearly  identifying 

the  offeror's  weaknesses  in  the  debriefing  benefits  the 

contractor  but  it  is  not  being  done  in  all  debrief ings. 

16.   Question  16.   The  intent  of  this  question  is  to 

determine  it  debrief ings  cover  the  three  major  elements  of 

most  proposals. 

a.   Question  to  unsuccessful  offerors 

The  Army  debriefs  me  on  the merits  of  my 

proposal  (circle  all  that  apply) . 

a.  technical 

b .  management 

c.  cost 

1.   Discussion 

This  question  allowed  the  respondents  to 

circle  all  answers  that  apply  to  debriefings  they  have 

attended.   A  total  of  113  answers  were  received  with  all  42 

offerors  selecting  at  least  two  answers.   Ninety  five  (95) 

percent  of  the  offerors  reported  that  the  Army  debriefs  them 

on  the  technical  merits  of  their  proposal.   Eighty  six  (86) 

percent  stated  that  they  are  debriefed  on  the  management 

merits  and  79%  said  they  are  debriefed  on  the  cost  merits  of 

their  proposal. 

b.   Question  to  AMC 

I  debrief  unsuccessful  offerors  on  the  

merits  of  their  proposal  (circle  all  that  apply) . 

a.  technical 

b .  management 


78 


c.   cost 

1.   Discussion 

The  question  was  set  up  to  allow  the 
contracting  officers  to  circle  all  answers  that  apply  to 
debrief ings  they  give.  All  32  contracting  officers  answered 
the  question  and  a  total  of  80  answered  were  recorded.  Eighty 
seven  (87)  percent  replied  that  they  debrief  the  technical 
merits  of  a  proposal.  Eighty  four  (84)  percent  stated  that 
they  debrief  the  management  merits  and  78%  said  that  they 
debrief  the  cost  merits  of  a  proposal .  Several  comments  were 
included  with  the  answers.  Three  people  wrote  that  they  cover 
all  criteria  advertised  in  the  RFP.  One  official  said  that  he 
also  debriefs  logistics,  production,  and  integrated  support. 
Another  contracting  officer  stated  that  he  covers  whatever 
issues  the  unsuccessful  offeror  wants  to  discuss.  Finally, 
one  respondent  stated  that  he  doesn't  debrief  cost  as  the 
unsuccessful  offeror  knows  his  price  and  the  winner's  price. 
c.   Analysis 

The  large  majority  of  both  the  unsuccessful 
offerors  and  contracting  officers  responded  that  debriefings 
cover  the  technical,  management  and  cost  merits  of  the 
proposal .  It  would  seem  though  that  these  answers  should  be 
approaching  100%  as  almost  all  procurements  that  warrant  a 
debriefing  contain  something  in  these  categories  to  evaluate. 
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This   is  certainly  true  of  price  as   it   is  a  required 

consideration  in  all  procurements.   The  procurement  command 

should  do  some  type  of  cost  realism  evaluation  on  each 

proposal  submitted.   To  not  debrief  price  indicates  that  the 

contracting  officer  is  not  passing  along  to  the  offeror  the 

evaluation  of  the  price  portion  of  the  proposal.   By  not 

debriefing  all  three  of  these  areas  contracting  officers  may 

leave   out   information   that   could   be   valuable   to   the 

unsuccessful  offeror. 

17.   Question  17  This  question  was  designed  to  determine 

if  the  unsuccessful  offerors  are  satisfied  with  the  debriefing 

on  the  technical,  management,  and  cost  portions  of  their 

proposal . 

a.   Question  to  unsuccessful  offerors 

I   am  satisfied  with   the  debriefing   on  the 
technical,  management,  and  cost  elements  of  my  proposal. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Forty  one  (41)  unsuccessful  offerors  answered 
this  question.  Five  (5)  percent  said  that  they  are  always 
satisfied  with  the  debriefing  on  the  technical,  management, 
and  cost  elements  of  their  proposal.  Eighty  (80)  percent 
stated  that  they  are  sometimes  satisfied  and  15%  responded 
that  they  are  never  satisfied  with  the  debriefing  on  these 
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elements.    Three  offerors  commented  that  debrief ings  are 

generic  in  nature  and  not  enough  detail  is  given. 

b.   Question  to  AMC 

Unsuccessful  offerors  are  satisfied  with  the 
debriefing  on  the  technical,  management,  and  cost  elements  of 
their  proposal . 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 

1.   Discussion 

Twenty  nine  (29)  contracting  officers  answered 
this  question.  Twenty  four  (24)  percent  stated  that 
unsuccessful  offerors  are  always  satisfied  with  the  debriefing 
on  the  technical,  management,  and  cost  elements  of  their 
proposal.  Sixty  six  (66)  percent  said  that  offerors  are 
sometimes  satisfied.  Three  (3)  percent  said  they  are  never 
satisfied  and  7%  stated  that  they  didn't  know.  Several 
comments  were  received.  Three  contracting  officers  stated 
that  the  unsuccessful  offerors  usually  feel  the  debriefing  is 
not  forthcoming  in  this  regard  because  the  offerors  want  to 
know  more  about  how  they  compared  with  the  winner's  proposal. 
One  contracting  officer  said  that  often  times  offeror's 
engineers  take  the  debrief  on  the  technical  aspects  of  the 
proposal  as  a  personal  attack  and  this  makes  it  extremely 
difficult  to  debrief  that  portion. 
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c.   Analysis 

A  small  percentage  of  both  groups  surveyed  stated 

that  unsuccessful  offerors  are  always  satisfied  with  the 

discussion  on  the  technical,  management,  and  cost  portions  of 

the  debriefing.   A  large  portion  of  each  said  offerors  are 

sometimes  satisfied.   This  is  most  likely  due  to  two  reasons. 

First,  the  offerors  are  looking  for  all  the  information  they 

can  get  concerning  the  evaluation  of  these  areas  of  their 

proposal  and  other  proposals,  especially  the  winners.  The  FAR 

and  AFARS  specifically  deny  access  to  some  of  the  information 

that  the  contractors  would  like  to  see.    Second,   the 

debriefing  may  not  contain  all  the  information  that  the  source 

selection  committee  has  on  the  proposal.  This  could  be  due  to 

poor  documentation,  a  briefing  from  a  second  hand  source  who 

is  not  as  knowledgeable  as  the  evaluator,  or  a  fear  of 

debriefing  something  that  was  discovered  and  not  addressed 

with  the  offeror  during  discussions. 

18.    Question  18   The  intent  of  this  question  is  to 

determine  if  unsuccessful  offerors  get  information  which  they 

can  use  in  drafting  proposals  for  future  procurements. 

a.   Question  to  unsuccessful  offerors 

Debrief ings  give  me  information  that  leads  to  more 
competitive  proposals  in  future  Government  competition. 

a .  always 

b.  sometimes 

c .  never 
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1.   Discussion 

Forty   one   (41)   offerors   answered   this 

question.   Seven  (7)  percent  stated  that  debrief ings  always 

give  them  information  which  leads  to  more  competitive 

proposals  in  future  procurements.   Seventy  one  (71)  percent 

reported  that  they  sometimes  get  information  to  aid  future 

proposals  and  22%  said  they  never  get  information  which 

improves  the  competitiveness  of  future  proposals.    Two 

comments  were  submitted  for  this  question.   One  contractor 

wrote  that  he  rarely  gets  this  information  from  the  debriefing 

but  through  "contacts"  in  the  community.   The  other  comment 

was  from  an  offeror  who  said  that  the  only  useful  information 

he  has  gotten  from  the  debriefings  is  that  the  low  cost 

proposal  always  seems  to  win.   As  a  result  he  is  turning  his 

best  value  proposals  into  low  cost  proposals. 

b.   Question  to  AMC 

Debriefings  give  unsuccessful  offerors  information 
that  leads  to  more  competitive  proposals  for  future  Government 
competition. 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 

1.   Discussion 

Thirty  (30)  contracting  officers  answered  this 
question.  Twenty  three  (23)  percent  stated  they  believe 
debriefings  always  give  information  to  unsuccessful  offerors 
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that  aid  them  in  preparing  more  competitive  proposals  in 
future  procurements.  Fifty  seven  (57)  percent  said 
debrief ings  sometimes  give  this  information.  Three  (3) 
percent  think,  debrief  ings  never  give  offerors  information  that 
is  useful  for  future  procurements,  and  17%  don't  know.  Three 
different  comments  were  submitted.  Six  contracting  officers 
stated  that  their  intent  is  to  give  out  information  that  will 
be  useful  for  future  competition.  One  official  wrote  that  the 
offerors  learn  how  AMC  utilizes  and  evaluates  best  value 
procurement.  If  they  don't  learn  anything  else  they  learn 
this.  The  last  person  to  comment  said  that  some  contractors 
accept  the  critique  and  criticism  of  their  proposal  and  learn 
from  the  debrief  and  some  argue  each  point  made  and  thus  are 
too  busy  arguing  and  defending  their  position  to  learn 
anything . 

c.   Analysis 

The  majority  of  those  surveyed  on  both  sides 
indicated  that  debriefings  sometimes  provide  information  that 
leads  to  more  competitive  proposals  in  the  future.  The  goal 
would  be  to  increase  the  information  that  allows  unsuccessful 
offerors  to  prepare  more  competitive  proposals  in  the  future. 
There  are  some  road  blocks  that  must  be  removed  to  improve 
this.  Unsuccessful  offerors  must  not  allow  their  emotions  or 
pride  to  interfere  with  assimilating  the  information  given  out 
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in  the  debriefing.   Arguing  each  point  brought  up  by  the 

contracting  officer  detracts  from  good  listening  and  may  put 

the  contracting  officer  on  the  defensive  and  cause  him  to 

refrain  from  passing  along  all  the  data  he  has.   Conversely 

contracting  officers  must  make  every  effort  to  get  as  much 

information  to  the  contractor  as  they  can. 

19.   Question  19  This  question  was  designed  to  determine 

if  after  a  debriefing  unsuccessful  offerors  fully  understand 

why  they  do  not  have  the  most  advantageous  proposal . 

a.   Question  to  unsuccessful  offerors 

Upon  conclusion  of   a  debrief,   I   completely 
understood  why  my  company  did  not  win  the  contract  award. 

a .  always 

b.  sometimes 

c.  never,  why  not 


1.   Discussion 

All  42  offerors  answered  this  question.  Two 
(2)  percent  of  the  contractors  reported  that  they  always 
completely  understood  why  they  didn't  win  the  contract  award. 
Seventy  six  (76)  percent  stated  that  they  sometimes  completely 
understood  and  22%  said  that  they  never  completely  understood 
why  they  didn't  win  the  contract  award.  Several  comments  were 
received  for  this  question.  Five  offerors  commented  that 
there  is  not  enough  information  given  out  to  completely 
understand  why  they  didn't  win  the  contract  award.  Three 
contractors  stated  that  they  often  feel  that  the  real  reasons 
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why  they  didn't  win  the  contract  award  are  not  given.  Another 

offeror  said  that  the  Army  doesn't  compare  proposals  so  you 

don't  know  if  you  lost  on  technical  merit  or  a  subjective  best 

buy. 

b.   Question  to  AMC 

Upon  conclusion  of  a  debrief  the  unsuccessful 
offeror  completely  understands  why  he  didn't  win  the  contract 
award 

a .  always 

b.  sometimes 

c .  never 

d.  don't  know 

1.   Discussion 

Thirty  one  (31)  contracting  officers  answered 
this  question.  Twenty  six  (26)  percent  said  they  believe  that 
after  a  debriefing  contractors  completely  understand  why  they 
didn't  win  the  contract  award.  Forty  two  (42)  percent  stated 
they  believe  contractors  sometimes  completely  understand. 
Three  (3)  percent  responded  that  contractors  never  understand 
and  29%  said  that  they  don't  know  if  a  contractor  completely 
understands  why  he  didn't  win  the  contract  award.  Several 
comments  were  submitted.  Two  officials  responded  that  a  lack 
of  details  of  the  winner's  proposal  or  any  comparative 
rankings  make  it  almost  impossible  for  the  offerors  to 
completely  understand  why  they  lost.  Three  people  wrote  that 
some  offerors  can't  or  refuse  to  recognize  that  someone  else's 
proposal  could  be  better  and  thus  can't  understand  how  they 
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lost.  One  contracting  officer  commented  that  the  regulatory- 
restrictions  on  what  can  be  discussed  sometimes  prevents  the 
offeror  from  getting  enough  information  to  completely 
understand  why  they  lost.  Finally  another  contracting  officer 
replied  that  when  the  offeror's  proposal  is  very  good  and  has 
no  real  weaknesses  but  the  winning  proposal  is  just  a  little 
better  it  may  be  very  difficult  for  the  offeror  to  understand 
why  they  didn't  win. 
c.   Analysis 

Contracting  officers  believe  the  unsuccessful 
offerors  understand  why  they  lost  the  contract  award  at  a 
higher  rate  than  the  unsuccessful  offerors  actually 
understand.  This  leads  to  the  conclusion  that  contracting 
officers  think  the  debriefings  are  more  effective  than  they 
actually  are.  Two  key  problems  are  identified  here.  First, 
unsuccessful  offerors  state  that  not  enough  information  is 
given  out  to  understand  why  they  didn't  win  the  contract 
award.  Second,  contracting  officers  commented  that  since 
comparisons  between  proposals  can't  be  made  it  makes  it 
difficult  for  contractors  to  get  enough  information  to 
completely  understand  why  they  lost.  The  Army  AFARS  thus  has 
a  negative  impact  on  the  quantity  of  information  the 
unsuccessful  offeror  can  get. 
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20.  Question  20.  The  intent  of  this  question  is  to 
determine  if  debriefings  are  of  value  to  the  unsuccessful 
offerors. 

a.   Question  to  unsuccessful  offerors 

The  debriefings  are  to  my  company. 

a.  valuable 

b.  somewhat  valuable 

c.  not  at  all  valuable 

1.   Discussion 

Forty  two  (42)  unsuccessful  offerors  answered 
this  question.  Thirty  eight  (38)  percent  stated  that  the 
debriefings  are  valuable,  50%  said  they  are  somewhat  valuable 
and  12%  reported  that  debriefings  are  not  at  all  valuable. 
Contractors  submitted  four  comments  to  this  question.  One 
contractor  wrote  that  although  debriefings  are  only  somewhat 
valuable  he  never  passes  up  an  opportunity  to  meet  with  his 
customer.  He  continued  that  team  building  and  getting  to  know 
the  key  players  are  too  important  to  not  attend  a  debriefing. 
Another  offeror  stated  that  the  real  details  on  the  evaluation 
of  his  proposal  are  dug  out  over  time  from  the  command,  thus 
making  the  debriefing  not  as  valuable  as  it  could  be.  Another 
comment  was  that  there  is  nothing  given  to  help  with  future 
proposals.  Finally  one  unsuccessful  offeror  responded  that 
the  debriefings  are  cordial  but  not  meaningful  in  terms  of 
useful  information  given  out. 
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b.   Question  to  AMC 

The  debriefings  I  give  are  to  unsuccessful 

offerors. 

a.  valuable 

b.  somewhat  valuable 

c.  not  at  all  valuable 

1.   Discussion 

Twenty  nine  (29)  contracting  officers  answered 
this  question.  Sixty  six  (66)  percent  believe  debriefings  are 
valuable  to  unsuccessful  offerors.  Thirty  four  (34)  percent 
believe  debriefings  are  somewhat  valuable  to  them  and  3% 
believe  debriefings  are  not  at  all  valuable  to  unsuccessful 
offerors.  Four  comments  were  received  with  the  answers.  One 
contracting  officer  stated  that  since  he  can't  compare  scores 
and  assessments  his  debriefings  are  not  as  useful  to 
unsuccessful  offerors  as  the  offerors  think  they  could  be. 
Another  contracting  officer  said  that  he  is  convinced 
debriefings  are  worthless  because  they  don't  tell  the 
unsuccessful  offerors  what  they  really  want  to  know-  how  they 
stood  compared  to  the  winner.  The  next  comment  was  that  the 
regulation  and  legal  constraints  sanitize  the  debriefing  so 
much  that  very  little  of  any  value  is  left  in  the  debriefing. 
Finally,  one  official  responded  that  all  the  unsuccessful 
offerors  want  to  talk  about  is  the  winner's  proposal  and  he 
doesn't  provide  much  information  on  it. 
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c.   Analysis 

Contracting  officers  believe  debriefings  are 

valuable  to  unsuccessful  offerors  at  a  much  higher  percentage 

than  unsuccessful  offerors  say  they  are.  Thus  debriefings  are 

not  as  valuable  as  contracting  officers  think  they  are. 

Consequently  many  contracting  officers  may  not  be  making  any 

attempts  to  improve  the  debriefing  process  as  they  believe  the 

system  is  satisfying  the  needs  of  the  unsuccessful  offerors. 

On  the  other  hand  the  contractors  need  to  understand  that  some 

information  that  they  would  like  to  get  debriefed  on  such  as 

details  of  the  winner's  proposal  is  not  releasable  under  the 

current  regulations. 

21.   Question  21  This  question  was  designed  to  determine 

if  AMC  offers  to  give  debriefings  to  unsuccessful  offerors 

that  don't  request  a  debriefing  on  their  own. 

a.   Question  to  unsuccessful  offerors 

The  Army  offers  to  debrief  me  even  when  I  don't 
ask  for  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Forty  two  (42)  unsuccessful  offerors  answered 
this  question.  Twelve  (12)  percent  stated  that  the  Army 
offered  to  debrief  them  even  when  they  didn't  ask  for  a 
debriefing.   Thirty  eight  (38)  percent  said  that  the  Army 
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sometimes  asked  them,  and  50%  responded  that  the  Army  never 

asked  them.  One  contractor  commented  that  the  notification  he 

receives  that  he  is  an  unsuccessful  offeror  typically  offers 

the  opportunity  for  a  debriefing. 

b.   Question  to  AMC 

My  organization  offers  to  debrief  all  unsuccessful 
offerors,  even  those  that  don't  request  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Thirty  one  (31)  contracting  officers  answered 
this  question.  Fifty  two  (52)  percent  responded  that  they 
always  offer  to  debrief  all  unsuccessful  offerors.  Thirteen 
(13)  percent  stated  that  they  sometimes  offer  to  debrief  those 
offerors  who  don't  ask  for  a  debriefing  and  35%  answered  they 
never  offer  to  debrief  offerors  that  don't  ask  for  a 
debriefing.  Several  comments  were  given  for  this  question. 
Six  contracting  officers  stated  that  they  never  offer  a 
debriefing  but  only  honor  requests.  One  official  commented 
that  at  the  major  system  level  debrief ings  are  normally  pre- 
scheduled  for  all  offerors.  Another  contracting  officer 
commented  he  offers  the  debrief  in  the  RFP  but  otherwise  it  is 
not  mentioned.  Finally,  one  official  responded  that  he  offers 
a  debriefing  in  the  letter  notifying  the  contractor  that  he 
didn't  win  the  contract  award. 
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c.   Analysis 

The  answers  between  the  two  groups  surveyed  vary. 

The  percentage  of  contracting  officers  who  say  they  offer 

debrief ings  to  all  unsuccessful  offerors,  even   those  that 

don't   ask   for  one,   is   higher  than  the  percentage   of 

unsuccessful  offerors  who  say  the  same  thing.  The  key  here  is 

that  offering  debriefings  to  all  unsuccessful  offerors  is  not 

happening.    The  regulations  do  not  require  AMC  to  offer 

debriefings  to  anyone.   It  only  states  that  debriefings  will 

be  given  to  those  unsuccessful  offerors  that  ask  for  one  in 

writing.  However,  if  debriefings  are  not  offered  there  may  be 

offerors  who  do  not  know  they  exist  or  that  they  can  get  one 

for  the  particular  procurement  they  submitted  a  proposal  for. 

22.   Question  22   This  question  was  designed  around  the 

specific  language  of  H.R.  3161.   The  intent  was  to  determine 

which  points  of  the  resolution  unsuccessful  offerors  and 

contracting  officers  like  and  to  determine  how  open  each  of 

the  two  groups  are  to  liberalizing  what  can  be  discussed  at 

debriefings.  Both  surveys  asked  for  the  respondents  to  circle 

all  answers  they  agreed  with. 

a.   Question  to  unsuccessful  offerors 

Debriefings   should  address   (circle  all   that 

apply) ,  as  long  as  confidential  business  information  is 

not  disclosed. 

a.    the  basic  proposed  technical  solution  of  the 
awardee 
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b.  the  overall  evaluated  cost  of  the  awardee  and 
debriefed  offeror 

c.  cost  or  price  associated  with  the  major  components 
of  the  awardee 's  proposal 

d.  overall  ranking  and  total  evaluation  scores  of  the 
awardee  and  debriefed  offeror 

e.  technical  point  scores  of  the  awardee  and 
debriefed  offeror 

f .  none  of  the  above 

g.  other 

1.   Discussion 

The  unsuccessful  offerors  showed  overwhelming 
support  for  answers  a  through  e.  They  felt  that  all  the 
changes  proposed  by  H.R.  3161  were  good  ideas.  Eighty  three 
(83)  percent  of  the  offerors  stated  that  debrief ings  should 
address  the  basic  proposed  technical  solution  of  the  awardee. 
Seventy  nine  (79)  percent  of  the  officials  said  debrief ings 
should  cover  the  overall  evaluated  cost  of  the  awardee  and 
debriefed  offeror.  Seventy  six  (76)  percent  of  the 
contractors  responded  that  the  cost  or  price  associated  with 
the  major  components  of  the  awardee 's  proposal  should  be 
addressed  in  debrief ings.  Ninety  (90)  percent  of  the 
unsuccessful  offerors  stated  that  the  overall  ranking  and 
total  evaluation  scores  of  the  awardee  and  debriefed  offeror 
should  be  covered.  Eighty  six  (86)  percent  of  the  contractors 
said  debriefings  should  contain  the  technical  point  scores  of 
the  awardee  and  debriefed  offeror.  Two  (2)  percent  of  the 
offerors  stated  that  debriefings  should  contain  none  of  the 
above.   Several  comments  were  submitted  with  these  answers. 
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Three  unsuccessful  offerors  stated  that  as  much  information  as 

can  be  disclosed  without  violating  proprietary  rights  should 

be  given  out.   Another  contractor  responded  that  a  narrative 

evaluation  of  the  debriefed  offeror's  proposal  should  be  given 

in  addition  to  the  other  answers.   One  offeror  said  that  the 

basic  rationale  for  selection  of  the  winning  proposal  should 

be  given.  Lastly,  one  contractor  stated  that  more  comparative 

information  should  be  given  out. 

b.   Question  to  AMC 

Debriefings   should  address   (circle  all   that 

apply) ,  as  long  as  confidential  business  information  is 

not  disclosed. 

a.  the  basic  proposed  technical  solution  of  the 
awardee 

b.  the  overall  evaluated  cost  of  the  awardee  and 
debriefed  offeror 

c.  cost  or  price  associated  with  the  major  components 
of  the  awardee 's  proposal 

d.  overall  ranking  and  total  evaluation  scores  of  the 
awardee  and  debriefed  offeror 

e.  technical  point  scores  of  the  awardee  and 
debriefed  offeror 

f .  none  of  the  above 

g.  other 

1.   Discussion 

Contracting  officers  didn't  show  much  support 
for  including  the  information  in  these  answers  in  debriefings. 
Thirteen  (13)  percent  of  the  contracting  officers  stated  that 
debriefings  should  address  the  basic  proposed  technical 
solution  of  the  awardee.  Forty  one  (41)  percent  of  the 
officials  said  the  overall  evaluated  cost  of  the  awardee  and 
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debriefed  offeror  should  be  included.  The  cost  or  price 
associated  with  the  major  components  of  the  awardee's  proposal 
should  be  covered  by  the  debriefing  according  to  6%  of  the 
contracting  officers.  Twenty  two  (22)  percent  of  these 
officials  said  that  the  overall  ranking  and  total  evaluation 
scores  of  the  awardee  and  debriefed  offeror  should  be  covered 
in  debrief ings.  Technical  point  scores  of  the  awardee  and 
debriefed  offeror  should  be  covered  in  debriefings  according 
to  3%  of  the  contracting  officers  surveyed.  Twenty  eight  (28) 
percent  of  the  contracting  officers  responded  that  none  of  the 
above  should  be  covered.  Sixteen  (16)  percent  of  the 
contracting  officers  stated  that  debriefings  should  contain 
other  items.  All  but  one  of  these  other  answers  stated  in 
general  terms  that  only  the  offeror's  proposal  should  be 
covered  and  that  weak  and  deficient  areas  should  be  addressed. 
In  other  words  these  officials  restated  what  the  FAR  requires. 
The  other  item  listed  was  that  the  basis  for  why  the  award  was 
given  to  the  winner  should  be  discussed.  Two  different 
comments  were  included  with  the  answers.  Two  contracting 
officers  said  that  information  regarding  the  winner's  proposal 
should  be  given  only  in  general  terms.  Another  contracting 
officer  stated  that  he  generally  gives  factor  ratings  and 
evaluated  cost  for  the  winning  proposal  and  factor,  sub- factor 
ratings  and  evaluated  cost  for  the  offeror's  proposal. 
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c.   Analysis 

The  unsuccessful  offerors  showed  strong  support 
for  debriefings  to  include  all  the  items  that  H.R.  3161 
proposed.  Contracting  officers,  on  the  other  hand,  were  much 
more  conservative  and  only  one  of  the  proposed  items  received 
higher  than  a  22%  support  of  the  contracting  officers.  This 
item  was  revealing  the  overall  evaluated  cost  for  the 
offeror's  and  winner's  proposals.  Almost  one  third  of  the 
contracting  officers  stated  that  none  of  the  answers  should  be 
addressed  in  debriefings.  The  answers  given  to  this  question 
show  that  unsuccessful  offerors  would  like  debriefings  to 
include  more  specific  information.  Additionally  this  question 
shows  the  conservative  nature  of  contracting  officers  with 
regard  to  debriefing.  Contracting  officers  didn't  show  strong 
support  for  any  of  these  proposed  additions  to  what  should  be 
covered  in  debriefings. 

23 .    Question  23    The  intent  of  this  question  is  to 
determine  if  when  a  debriefing  is  conducted  has  any  impact  on 
if  an  unsuccessful  offeror  will  protest  the  contract  award. 
a.   Question  to  unsuccessful  offerors 

The  sooner  I'm  debriefed  the  less  likely  I  am  to 


protest 


a .  always 

b.  sometimes 

c .  never 
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1.   Discussion 

Forty  one  (41)  unsuccessful  offerors  answered 

this  question.  Twelve  (12)  percent  answered  it  is  always  true 

that  the  sooner  they  are  debriefed  the  less  likely  they  are  to 

protest.   Fifty  one  (51)  percent  stated  that  sometimes  the 

sooner  they  are  debriefed  the  less  likely  they  are  to  protest. 

Fifteen  (15)  percent  of  the  offerors  stated  that  this  is  never 

the  case.   Twenty  two  (22)  percent  didn't  answer  one  of  the 

possible  answers  but  commented  the  question  was  not  applicable 

to  them.    This  may  mean  they  never  protest  or  that  the 

timeliness  of  a  debriefing  doesn't  impact  on  their  decision  to 

protest.    One  offeror  commented  that  the  quality  of  the 

debriefing  and  not  the  time  when  it   is  given  is  the 

determining  factor  if  a  protest  will  be  filed.  Three  offerors 

stated  they  never  protest. 

b.   Question  to  AMC 

The  sooner  a  debriefing  is  conducted,  the  more 
likely  it  is  to  prevent  a  protest. 

a .  always 

b.  sometimes 

c .  never 

1.   Discussion 

Twenty  eight  (28)  contracting  officers 
answered  this  question.  Thirty  two  (32)  percent  stated  that 
the  sooner  a  debriefing  is  conducted  the  more  likely  it  is  to 
prevent  a  protest.  Fifty  seven  (57)  percent  said  that  this  is 
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sometimes  the  case  and  10%  responded  that  this  is  never  the 
case.  Several  comments  were  submitted  with  the  answers.  Four 
contracting  officers  commented  that  they  don't  think  when  a 
debriefing  is  done  influences  if  an  unsuccessful  offeror  will 
protest.  Another  contracting  officer  stated  that  the 
unsuccessful  offeror  only  has  10  days  to  file  a  protest  to 
stop  the  procurement  action. 
c.   Analysis 

Many  unsuccessful  offerors  and  contracting 
officers  say  that  the  sooner  a  debriefing  is  conducted  the 
less  likely  the  unsuccessful  offeror  is  to  protest.  This 
should  certainly  be  true  for  debrief ings  that  are  done  within 
the  window  authorized  for  the  unsuccessful  offeror  to  submit 
a  protest.  A  competent  debriefing  conducted  within  this 
window  that  shows  the  source  selection  process  was  proper  and 
fair  should  yield  few  if  any  protests.  A  debrief  conducted 
after  the  10  day  protest  window  may  initiate  a  protest  because 
the  offeror  may  have  questions  concerning  the  fairness  of  the 
source  selection  process  but  doesn't  have  the  information  that 
the  debriefing  would  provide  that  shows  the  selection  process 
was  proper. 

24.  Question  24  The  intent  of  this  question  is  to 
determine  what  the  strengths  of  the  Army  debriefing  process 
are.  This  question  is  valuable  because  the  set  up  and  content 
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of  a  debriefing  are  very  dependent  on  which  command  the 
debriefing  is  done  in  and  which  contracting  officer  conducts 
the  debriefing.   Debrief ings  in  each  command  can  be  improved 
by  incorporating  the  strengths  from  the  other  commands. 
a.   Question  to  unsuccessful  offerors 

List  3  strengths  of  the  Army  debriefing  process. 

1.   Discussion 

Twenty  six  (26)  of  the  42  unsuccessful 
offerors  who  returned  surveys  answered  this  question.  Many 
strengths  of  the  debriefing  process  were  identified.  The  major 
themes  from  these  strengths  are  listed  below.  The  number  of 
unsuccessful  offerors  that  identified  each  area  as  a  strength 
are  annotated  in  parenthesis. 

•  Contracting   officers   offer   to   debrief   unsuccessful 
offerors  (2) . 

•  There  are  no  restrictions  on  who  may  attend  (2) . 

•  The  debrief ings  are  done  in  a  timely  manner  (8) . 

•  The  evaluators  are  normally  at  the  debriefing  and  they 
debrief  their  part  or  are  available  to  answer  questions 

(4)  . 

•  Debriefings  are  well  organized  and  generally  follow  the 
RFP  outline  (2)  . 

•  Debriefings  allow  for  interaction  between  the  two  parties 
and  both  sides  get  to  know  each  other  (2) . 

•  Personnel  who  debrief  are  always  prepared,  many  times  with 
a  script  (2) . 
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•  Debrief ings  isolate  major  shortcomings  and  weaknesses  in 
each  area  evaluated  (4) . 

•  Strengths  of  our  proposal  are  identified  along  with  our 
weaknesses  (2) . 

•  If  the  debriefing  is  conducted  properly  it  will  help 
improve  the  competitiveness  of  future  proposals  (1) . 

•  Debrief ings  are  thorough  (2) . 


b.   Question  to  AMC 

List  3  strengths  of  the  Army  debriefing  process. 

1.   Discussion 

Twenty  one  (21)  contracting  officers  answered 
this  question.  Many  strengths  of  the  debriefing  process  were 
identified  by  the  contracting  officers.  The  major  themes  from 
these  strengths  follow: 


•  Debriefings  provide  an  opportunity  for  contracting 
officers  to  give  information  to  unsuccessful  offerors  that 
will  strengthen  their  future  proposals  (8) . 

•  Debriefings  are  given  in  a  timely  manner  (4) . 

•  Debriefings  show  the  fairness  and  integrity  of  the  source 
selection  process  (6) . 

•  Debriefings  add  accountability  to  the  evaluation  process 

(l)  . 

•  Debriefings  answer  all  questions  relative  to  the  offeror's 
proposal  (1) . 

•  Debriefings  reveal  the  strengths  and  weaknesses  of  the 
offeror's  proposal  (5). 

•  Debriefings  give  a  lot  of  details  to  the  unsuccessful 
offeror  regarding  the  evaluation  of  his  proposal  (4) . 
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Contracting  officers  are  willing  to  discuss  all  issues 
pertinent  to  the  evaluation  of  the  proposal  (1) . 

Debrief ings  explain  the  basis  for  the  award  (1) . 


c.   Analysis 

The  survey  results  show  that  unsuccessful  offerors 
and  contracting  officers  believe  there  are  many  strengths  to 
the  debriefing  process.  All  the  items  given  as  strengths 
should  benefit  the  debriefing  process.  The  majority  of  the 
strengths  listed  were  mentioned  by  four  or  fewer  survey 
participants.  This  could  indicate  that  debrief ings  are 
conducted  somewhat  differently  by  different  commands  and 
contracting  officers  and  that  the  parties  involved  in  the 
debriefing  process  have  differences  of  opinion  concerning  what 
is  important  to  the  debriefing  process. 

25.   Question  25   This  question  was  designed  to  identify 
the  weaknesses  of  the  debriefing  process.    Once  these 
weaknesses  are  identified  steps  can  be  taken  to  correct  these 
problems.   This  will  improve  the  debriefing  process. 
a.   Question  to  unsuccessful  offerors 

List  3  weaknesses  of  the  Army  debriefing  process 

1.   Discussion 

Thirty  (30)  unsuccessful  offerors  answered 
this  question.  Many  weaknesses  were  listed.  The  major  themes 
of  the  weaknesses  follow: 
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•  Debrief ings  are  not  done  in  a  timely  manner  (5) . 

•  Debrief ings  do  not  provide  enough  details  (9) . 

•  Personnel  who  debrief  are  very  cautious  in  what  they  say 
(4)  . 

•  Limited  information  is  given  out  due  to  the  restrictions 
imposed  by  the  regulations  (6) . 

•  Debriefings  offer  very  little  information  useful  for 
future  proposals  (3). 

•  No  information  is  given  on  the  awardee's  proposal  (3) . 

•  The  main  players  in  the  source  selection  are  sometimes  not 
in  attendance  (2) . 

b.   Question  to  AMC 

List  3  weaknesses  of  the  Army  debriefing  process. 

1.   Discussion 

Seventeen  (17)  contracting  officers  answered 
this  question.  They  stated  there  were  many  weaknesses  in  the 
debriefing  process.  The  major  weaknesses  they  indicated 
follow: 


•  Comparisons  of  proposals  must  be  done  to  make  the 
debriefing  meaningful  (3) . 

•  There  are  limits  on  the  release  of  information  due  to  the 
regulations  (2) . 

•  Debriefing  can't  cover  any  aspects  of  the  winner's 
proposal  except  his  total  price  (2) . 

•  Fear  of  protests  causes  contracting  officers  to  control 
what  is  said  in  the  debrief.  Sometimes  the  debrief  is 
even  scripted  (3) . 
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•  Key  players  in  the  source  selection  process  are  sometimes 
not  available  for  the  debriefing  (1) . 

•  Often  more  is  revealed  in  a  debriefing  than  is  required 
(1)  . 


c.   Analysis 

The  research  shows  there  are  several  weaknesses  to 

the  debriefing  process.   Every  one  of  the  items  identified  as 

a  weakness  should  have  an  adverse  impact  on  the  debriefing 

process.   All  but  two  of  the  weaknesses  given  involved  the 

lack  of  information  given  out  in  debrief ings.  This  indicates 

that  a  major  deficiency  with  the  debriefing  process  is  the 

lack  of  information  released  to  unsuccessful  offerors. 

26.   Question  2  6 

a.   Question  to  unsuccessful  offerors  The  intent  of 

this  question  is  to  determine  if  unsuccessful  offerors  ask  a 

core  set  of  similar  questions  at  debrief  ings.   If  they  do, 

contracting  officers  can  prepare  for  these  questions  and 

unsuccessful  offerors  may  then  get  better  answers  to  these 

common  questions. 

The  3  most  common  questions  I  ask  during  a 
debriefing  are  . 

1.   Discussion 

Thirty  three  (33)  unsuccessful  offerors 
answered  this  question.  A  total  of  74  questions  were  given. 
Many  respondents  listed  only  one  or  two  questions  for  their 
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answer.  Questions  that  were  listed  at  least  three  times  are 
considered  common  questions.  The  following  nine  questions 
appeared  at  least  three  times. 

•  What  was  the  evaluation  of  our  technical  approach  (12)? 

•  What  are  the  weaknesses  in  our  proposal  (8)? 

•  What  was  the  evaluation  of  our  cost  (8)? 

•  What  was  the  winner's  proposal  and  evaluation  of  that 
proposal  (6) ? 

•  How  did  we  compare  with  the  awardee  (5)? 

•  What  are  the  strengths  of  our  proposal  (4)? 

•  How  did  the  evaluators  judge  our  risk  (3)? 

•  What  was  our  overall  ranking  (3)? 

•  What  should  have  I  done  to  have  won  this  competition  (3)? 

2 .   Analysis 

There  are  many  similar  questions  that  are 
asked  by  unsuccessful  offerors  during  debrief ings.  One  of 
these  questions  cannot  be  answered  according  to  the  guidance 
in  the  AFARS  and  another  question  can  only  be  answered  in  a 
very  general  nature  in  accordance  with  the  FAR.  The  AFARS 
states  that  comparisons  will  not  be  made  between  proposals  [1: 
p. 10 -3].  Therefore  unsuccessful  offerors  will  not  get  an 
answer  to  their  question  of  how  did  they  compare  to  the  winner 
unless  the  contracting  officer  deviates  from  the  AFARS 
guidance.  The  FAR  states  that  debrief ings  will  not  reveal  the 
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relative  merits  or  information  covered  by  the  Freedom  of 

Information  Act  on  competitors  proposals  [7:  p. 15-41].   This 

will  severely  limit  what  can  be  discussed  in  response  to  the 

question  what  was  the  winner's  proposal  and  evaluation  of  that 

proposal.   The  other  questions  can  be  thoroughly  answered  by 

debriefing  personnel.  Contracting  officers  that  are  prepared 

to  answer  these  questions  will  give  better  answers  to  these 

questions  than  those  who  are  not  ready  to  answer  them. 

c.    Question  to  AMC    This  question  was  designed  to 

determine  if  AMC  MSC's  have  internal  instructions  that  give 

guidance  to  contracting  officers  on  debriefing  unsuccessful 

offerors. 

My  organization  has  internal  instructions  covering 
debriefing  procedures. 

a.  yes 

b.  no 

1.   Discussion 

Thirty  one  (31)  contracting  officers  answered 
this  question.  Forty  five  (45)  percent  stated  that  their 
organization  has  internal  instructions  covering  debriefing 
procedures.  Fifty  five  (55)  percent  of  the  contracting 
officers  responded  their  command  doesn't  have  internal 
instructions  on  debriefing  procedures. 
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2 .   Analysis 

Results  of  this  question  show  that  not  all 
commands  have  guidance  for  debriefing  unsuccessful  offerors. 
Lack  of  any  clear  guidance  as  to  what  may  be  covered  in  a 
debriefing  and  what  may  not  can  cause  contracting  officers  to 
take  a  very  conservative  view  of  the  general  guidance  in  the 
FAR  and  AFARS .  Follow  up  telephone  calls  to  the  five  MSC's 
surveyed  revealed  that  three  MSC's  have  written  guidance 
concerning  debrief ings.  Only  one  of  these  has  any  specific 
guidelines.  This  MSC's  internal  instructions  has  several 
recommendations  for  what  should  be  covered  in  a  debriefing. 
It  states  that  the  contractor  officer  should  consider  the 
following  items  in  the  debriefing:  a  general  explanation  of 
the  source  selection  organization,  the  basis  for  the  award, 
the  ratings  the  offeror's  proposal  received  on  all  the  factors 
and  sub- factors,  an  explanation  of  why  each  rating  was  given, 
the  advantages  and  disadvantages  of  the  proposal ,  and  a  brief 
explanation  of  why  the  successful  offeror  won.  The 
explanation  of  why  the  winner  won  could  include  showing  the 
winner's  ratings  for  each  factor  and  sub-factor  without  any 
detailed  explanation  of  each  rating.  This  guidance  adds  some 
structure  to  the  debriefing  process.  It  also  provides  some 
framework  for  what  can  be  discussed  about  the  winner's 
proposal  evaluation.  This  can  add  valuable  information  to  the 
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debriefing,  as  the  unsuccessful  offeror  can  see  where  his 

proposal  stood  in  relation  to  the  winner's  in  terms  of  the 

overall  rating  for  each  area.    The  guidance  this  MSC's 

internal  instructions  provide  allow  contracting  officers  who 

have  interpreted  the  FAR  and  AFARS  conservatively  to  give 

information  that  they  may  not  have  given  absent  this  guidance. 

27 .   Question  27   The  intent  of  this  question  was  to  have 

both  the  unsuccessful  offerors  and  contracting  officers  offer 

their  ideas  on  things  the  Army  could  do  to  improve  the 

debriefing  process. 

a.   Question  to  unsuccessful  offerors 

The  Army  could  improve  the  debriefing  process 
by • 


1.   Discussion 

Twenty  eight  (28)  unsuccessful  offerors 
answered  this  question.  Many  proposals  were  provided  to 
improve  the  debriefing  process.  The  major  themes  of  these 
proposals  follow: 

•  Allow  some  comparisons  with  the  winning  proposal  (7) . 

•  Improve  evaluation  process  so  that  the  people  debriefing 
can  give  an  honest  and  detailed  debriefing  (5) . 

•  Be  more  specific  when  discussing  deficiencies  (4) . 

•  Provide  more  information  on  winning  proposal  (4) . 

•  Approach  process  in  a  more  positive  way  (3) . 

•  Provide  read  ahead  packets  (2) . 
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Debrief ings  should  be  done  in  a  more  timely  manner  (2) . 

Discuss  strengths  as  well  as  weaknesses  (2) . 

Allow  contractors  to  submit  questions  before  debriefing  so 
answers  can  be  better  researched  (1) . 

Reveal  how  unsuccessful  offeror  ranked  in  all  areas 
(1)  . 

Provide  rating  and  risk  assessment  for  each  evaluated  area 
for  both  the  unsuccessful  offeror's  and  winner's  proposal 
(1). 

Release  SSA  decision  document  (1) . 

b.   Question  to  AMC 

I/The  Army  could  improve  the  debriefing  process  by 


1.   Discussion 

Twenty  two  (22)  contracting  officers  answered 
the  question.  Many  answers  were  given.  The  major  themes  for 
improving  the  debriefing  process  follow: 

•  Issue  some  direction  other  than  what  is  in  the  FAR  (4) . 

•  Write  solicitations  and  evaluation  criteria  as  precise  as 
possible  to  make  evaluations  and  subsequent  debriefings 
easier  (2) . 

•  Ask  for  questions  in  advance  (1) . 

•  Give  unsuccessful   offerors   their  evaluation  ratings 

(1)  . 

•  Establish  guidelines  for  consistency  of  releasable 
information  (1)  . 

•  Allow  comparisons  between  unsuccessful  offeror's  proposal 
and  winning  proposal  (2) . 
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•  Give  unsuccessful  offerors  debriefing  charts  in  advance  of 
the  debriefing  (1)  . 

•  Provide  maximum  information  allowed  (3) . 

•  Contracting  officers  should  be  more  open  and  honest 
(1)  . 

•  Give  unsuccessful  offerors  source  selection  decision 
memorandum  (1) . 

•  Hold  debrief ings  later  than  they  are  now  held  (2) . 


c.   Analysis 

The  results  of  the  research  show  there  are  many 
ways  that  contracting  officers  and  AMC  could  improve  the 
debriefing  process.  Several  of  the  recommended  ways  to 
improve  debriefings  were  mentioned  by  both  unsuccessful 
offerors  and  AMC  officials.  These  areas  were  allowing  some 
comparisons  with  the  winning  proposal,  give  out  debriefing 
packets  and  do  so  before  the  debriefing,  provide  offerors  with 
their  ratings  for  the  evaluated  areas,  provide  as  much 
information  as  possible,  and  releasing  the  source  selection 
decision  document.  One  recommendation  that  the  two  survey 
groups  disagreed  on  was  when  to  conduct  debriefings. 
Contractors  stated  that  the  debriefing  process  could  be 
improved  by  holding  debriefings  in  a  more  timely  manner  while 
AMC  officials  said  to  improve  the  debriefings  they  should  be 
held  later  than  they  are  now  held. 
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28.  Question  28.  This  question  was  designed  to  get 
recommendations  from  unsuccessful  offerors  and  contracting 
officers  of  how  unsuccessful  offerors  could  improve  the 
debriefing  process. 

a.  Question  to  unsuccessful  offerors 

I/unsuccessful   offerors   could   improve   the 
debriefing  process  by  . 

1.   Discussion 

Twenty  four  (24)  unsuccessful  offerors 
answered  this  question.  The  major  themes  of  their  proposals 
follow: 

•  Submit  questions  before  the  debriefing  (7) . 

•  Have  the  right  personnel  there  (3) . 

•  Approach  the  debriefing  with  a  positive  attitude  (5) . 

•  Limit   protests   to   procurements   that   show   a   clear 
impropriety  or  prejudice  (2) . 

•  Ask  as  many  questions  as  needed  but  ensure  they  are 
relevant  and  appropriate  (6) . 

b.  Question  to  AMC 

Unsuccessful  offerors  could  improve  the  debriefing 
process  by  . 


1.   Discussion 

Twenty  one  (21)  contracting  officers  answered 
the  question.  Several  themes  were  given  as  possible  ways  to 
improve  the  system: 
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•  Accept  source  selection  and  don't  protest  unless 
impartiality  is  displayed  (3) . 

•  Be  receptive  rather  than  defensive  (2) . 

•  Don't  challenge  the  source  selection  decision  (3). 

•  Personnel  in  attendance  should  have  a  good  understanding 
of  evaluation  factors  and  proposal  (2) . 

•  Unsuccessful  offerors  should  ask  questions  in  advance 

(2)  . 

•  Unsuccessful  offerors  should  ensure  their  decision  makers 
for  technical  and  cost  areas  are  in  attendance  (3) . 

•  Unsuccessful  offerors  should  be  familiar  with  regulatory 
restrictions  concerning  debrief ings  (1) . 

•  Unsuccessful  offerors  should  not  be  so  emotional  (2) . 

c.   Analysis 

The  results  of  the  survey  indicate  there  are 
several  ways  that  unsuccessful  offerors  could  improve  the 
debriefing  process.  Four  of  these  recommendations  were  made 
by  both  groups  of  respondents.  They  are:  submit  questions 
before  the  debriefing,  have  the  right  personnel  attend  the 
debriefing,  approach  the  debriefing  with  the  right  frame  of 
mind,  and  limit  protests  to  procurements  that  show  a  clear 
impropriety  or  unfairness  in  the  source  selection  process. 
Two  of  these  recommendations  could  be  difficult  for  all 
contractors  to  execute.  Approaching  a  debriefing  with  the 
right  frame  of  mind  by  being  receptive  rather  than  defensive 
and  accepting,  not  challenging  the  source  selection  can  be 
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hard  to  do.  It  is  natural  for  a  contractor  to  be  defensive 
when  his  proposal  was  not  selected  as  the  winning  proposal  and 
then  attend  a  debriefing  where  someone  tells  him  all  the 
things  that  were  wrong  with  the  proposal.  It  is  also  natural 
for  contractors  to  think  they  submitted  the  best  proposal  and 
the  only  way  they  could  have  lost  was  if  the  source  selection 
team  somehow  made  a  mistake. 

C.   INTERVIEWS 
1 .   General 

Interviews  were  done  with  defense  contractors  and 
contacting  officers  as  a  follow  up  to  the  surveys.  The 
interviews  were  done  with  three  defense  contractors  to  get 
their  opinions  on  how  the  debriefing  process  could  be 
improved.  These  interview  participants  included  a  large 
hardware  company,  a  large  services  company  and  a  small 
hardware  company.  The  intent  of  the  interviews  was  to  have  a 
discussion  and  ask  questions  to  illicit  responses  on  how  to 
improve  the  debriefing  process  that  may  have  been  missed  by 
the  survey  participants.  Contracting  officers  from  three 
MSC's  within  AMC  were  then  interviewed  to  get  their  opinions 
on  the  feasibility  of  implementing  the  major  suggested  changes 
to  the  debriefing  process. 


112 


2.   Interview  Results 

The  three  defense  contractors  suggested  a  total  of  19 
changes  to  the  current  debriefing  process  that  they  felt  would 
improve  debrief ings.  All  but  four  of  these  changes  were 
identified  by  the  survey  participants.  The  contracting 
officers  that  were  interviewed  believe  14  of  these 
recommendations  could  be  implemented. 

Recommendations  which  were  identified  in  both  the 
surveys  and  interviews  that  would  improve  the  debriefing 
process  are: 

•  Debrief ings  should  be  done  in  a  timely  manner. 

•  AMC  MSC's  should  not  place  any  limits  on  the  employees 
that  may  attend  the  debriefing  from  the  unsuccessful 
offerors. 

•  Contracting  officers  should  debrief  in  an  outline  similar 
to  the  RFP. 

•  Contracting  officers  should  give  unsuccessful  offerors  a 
debriefing  packet  and  do  so  before  the  debrief  if 
possible. 

•  Unsuccessful  offerors  should  submit  questions  in  advance 
of  the  debriefing. 

•  SSEB  members  and  any  other  evaluators  should  be  in 
attendance  at  the  debriefing  to  participate  in  the  debrief 
and  answer  questions. 

•  Debrief ings  should  cover  both  the  strengths  and  weaknesses 
of  the  offeror's  proposal. 

•  Debriefings  should  cover  the  ratings  of  the  evaluated 
areas  for  both  the  offeror's  and  winner's  proposals. 
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•  Narrative  comments  should  cover  all  evaluated  areas  and  be 
specific. 

•  General  details  of  the  winner's  proposal  evaluation  should 
be  discussed. 

•  Some  comparisons  between  unsuccessful  offeror's  and 
winner's  proposals  should  be  revealed. 

•  Contracting  officers  should  discuss  the  basis  for  the 
award  decision. 

•  Some  of  the  restrictions  that  the  regulations  stipulate 
concerning  what  shouldn't  be  revealed  in  debrief ings 
should  be  lifted. 

•  Specific  guidance  on  what  should  and  shouldn't  be  covered 
in  debrief ings  should  be  developed. 

•  The  SSA  decision  document  should  be  released  to 
unsuccessful  offerors. 

Contracting   officers   believe   that   several   of   these 

recommendations  are  not  feasible.  They  believe  that  handouts 

should  be  given  out  to  the  unsuccessful  offerors.   This  will 

allow  the  offerors  to  listen  more  closely  to  the  debrief 

rather  than  trying  to  write  down  what  they  hear  and  see. 

However  it  would  be  difficult  to  give  these  handouts  to  the 

unsuccessful  offerors  before  the  debriefing  if  the  debrief ings 

are  conducted  soon  after  the  contract  award.   Some  time  is 

needed  to  prepare  handouts  for  all  the  unsuccessful  offerors 

and  there  would  not  be  enough  time  to  prepare  all  these 

handouts  far  enough  in  advance  to  send  them  to  the  offerors 

[22]  . 
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Next,  contracting  officers  believe  that  ratings  for 
the  winner's  proposal  should  not  be  given  out  in  the 
debriefing  [21].  They  feel  that  discussing  the  winner's 
ratings  would  provide  too  much  information  on  the  winner's 
proposal . 

Providing  the  general  details  of  the  winner's  proposal 
evaluation  to  the  unsuccessful  offeror's  is  inappropriate 
according  to  contracting  officers  [22.]  .  Some  details  on  the 
winner's  cost  or  price  can  be  released  but  this  is  the  only 
information  that  should  be  released.  Contracting  officers 
further  stated  that  if  unsuccessful  offeror's  want  any 
additional  information  on  the  winner's  proposal  they  should 
request  it  under  the  Freedom  of  Information  Act  [2_1]  . 

Finally,  contracting  officers  interviewed  believe  that 
specific  guidance  on  what  should  and  shouldn't  be  covered  in 
debrief ings  should  not  be  developed.  They  feel  they  have  the 
latitude  to  work  within  the  spirit  of  the  regulations  to 
tailor  debrief ings  as  needed  depending  on  the  situation  [21] . 

The  remaining  four  recommendations  were  identified  in 
the  interviews  only.  The  first  of  these  recommendations  is 
that  debriefings  should  be  offered  to  all  unsuccessful 
offerors  [14] .  Offering  a  debriefing  shows  courtesy  to  the 
unsuccessful  offeror.  The  contractor  spent  precious  resources 
on  the  proposal  and  offering  to  debrief  him  shows  the 
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contracting  command  appreciates  the  offeror's  attempt  to  do 
business  with  the  command.  Additionally  the  offer  to  debrief 
all  offerors  shows  the  contractors  that  the  contracting 
officer  has  nothing  to  hide  and  that  the  source  selection  was 
done  properly.  Finally,  offering  a  debriefing  to  all 
unsuccessful  offerors  will  ensure  that  those  contractors  who 
are  unaware  that  they  are  entitled  to  a  debriefing  know  they 
can  receive  one.  There  is  nothing  that  prevents  contracting 
officers  from  offering  to  debrief  all  unsuccessful  offerors 
and   some   contracting   officers   already   do   this   [21] . 

Debriefing  all  evaluated  areas  would  also  improve  the 
debriefing  process  [13.]  .  A  debriefing  that  covers  all 
evaluated  areas  allows  the  unsuccessful  offeror  to  understand 
where  his  proposal  stood  in  relation  to  all  evaluation 
criteria.  This  type  of  debriefing  would  be  superior  to  one 
where  the  offeror  was  only  debriefed  in  certain  areas  that  he 
was  particularly  weak  in,  as  covering  all  areas  would  provide 
a  broader  base  of  information  about  the  entire  proposal. 

Another  defense  contractor  stated  that  not  only  should 
each  evaluated  area  in  the  debrief  be  discussed  but  the  sub- 
factors  in  each  area  should  be  briefed  [13.]  .  Debrief ings  that 
address  each  sub- factor  area  should  be  much  more  informative 
than  debriefings  that  address  only  evaluation  factors  or 
general  areas.   Offerors  would  get  an  understanding  of  how 
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their  proposal  stood  up  to  the  evaluation  criteria  on  a  much 
more  specific  basis. 

Revealing  the  rankings  for  each  evaluated  area  for 
both  the  unsuccessful  offeror's  and  winner's  proposals  would 
also  improve  the  debriefing  process  [14]  .  This  would  allow 
contractors  to  see  where  they  stood  in  relation  to  the  other 
offerors  and  the  winner.  This  information  cannot  be  released 
for  technical  areas,  however,  according  to  the  FAR  [7:  p.  15- 
41]  .  Additionally  giving  out  this  information  for  other  areas 
could  be  considered  making  point  by  point  comparisons,  which 
the  AFARS  prohibits.  Revealing  the  rankings  for  the 
unsuccessful  offeror's  and  winner's  evaluated  areas  would  be 
inappropriate  according  to  contracting  officers.  Contracting 
officers  stated  this  would  be  providing  point  by  point 
comparisons  which  is  prohibited  by  the  AFARS  [22] . 

D.   SUMMARY 

1.   Debrief ings  within  AMC 

The  results  of  the  survey  provide  an  overview  of  what 
a  debriefing  is  like  within  AMC.  However,  as  the  survey 
results  show  no  two  debrief ings  are  exactly  the  same.  There 
are  too  many  variables  that  can  be  changed  to  effect  the 
debriefing  process.  These  variables  include  things  such  as 
who  the  contracting  officer  is,   the  size  and  type  of 
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procurement,  the  number  and  complexity  of  evaluation  criteria 
used,  the  detail  of  the  evaluation,  the  experience  of  the 
personnel  involved,  and  the  concerns  and  questions  of  the 
unsuccessful  offerors.  This  section  summarizes  the  answers  to 
the  survey  which  show  what  a  generic  debriefing  of  an 
unsuccessful  offeror  is  within  AMC.  It  will  discuss  when  and 
where  done,  who  attends,  and  what  is  discussed.  Because  no 
two  debrief ings  are  the  same,  debrief ings  within  AMC  may  vary 
somewhat  from  the  following  baseline  description. 

Debriefings  must  be  conducted  for  those  unsuccessful 
offerors  that  request  a  debriefing  in  writing.  This  is  in 
accordance  with  the  FAR.  The  number  of  unsuccessful  offerors 
that  request  a  debriefing  is  generally  quite  high.  More  than 
two  thirds  request  a  debriefing  with  this  number  being  even 
higher  for  major  acquisitions  or  best  value  type  procurements. 
In  addition,  many  contracting  officers  offer  to  debrief  the 
unsuccessful  offerors  even  if  the  offeror  does  not  ask  for  a 
debriefing.  This  is  not  required  by  any  regulation  but  is 
done  out  of  courtesy  for  the  unsuccessful  offerors.  Many 
contractors  will  accept  this  offer  of  a  debrief.  This 
increases  the  percentage  of  contractors  being  debriefed  to  a 
number  higher  than  those  that  would  have  been  debriefed  from 
a  contractor  initiated  request. 
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Two  regulations  provide  guidance  on  when  debriefings 
will  be  conducted.  The  FAR  states  that  offerors  will  be 
debriefed  as  soon  as  possible  after  their  request.  AMC  P  715- 
3  Vol.  1  states  that  debriefings  will  be  held  after  contract 
award  [2:  p. 51] .  Contracting  officers  hold  debriefings  as 
soon  as  possible  after  their  request.  Contracting  officers 
report  this  is  usually  within  ten  days  of  the  request. 
Unsuccessful  offerors  dispute  this  and  report  that  as  the 
majority  say  debriefings  are  held  between  11  and  20  days  of 
request. 

There  is  no  written  guidance  on  where  debriefings  are 
to  be  held.  However,  most  debriefings  are  in  person  and  it  is 
standard  practice  that  debriefings  are  held  at  the  contracting 
officer's  command.  If  the  contractor  wants  to  attend  a 
debriefing  it  is  incumbent  on  him  to  travel  to  the  contracting 
officer's  location.  Sometimes  a  telephone  debriefing  may  be 
conducted  but  this  is  done  only  at  the  unsuccessful  offeror's 
direction.  Debriefing  by  phone  is  normally  done  when  the 
contract  is  small  or  the  debriefing  is  easily  understandable 
by  phone.  In  these  cases  it  is  not  cost  effective  for  a 
contractor  to  pay  travel  costs  to  attend  an  in  person 
debriefing. 

The  number  of  people  who  attend  the  debriefing  is 
dependent  on  the  size  and  complexity  of  the  procurement.   For 
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large  or  complex  procurements  representatives  of  AMC  may 
include  the  source  selection  evaluation  board  chairman,  the 
factor  chairmen,  the  contracting  officer,  contracting 
specialist,  and  a  legal  representative.  Any  other  special 
support  personnel  are  added  as  needed.  Representatives  from 
the  unsuccessful  firm  that  may  attend  a  debriefing  for  large 
or  complex  procurements  may  include  the  vice  president, 
program  manager,  proposal  manager,  and  contracts  manager.  For 
smaller  or  less  complex  procurements  there  would  generally  be 
fewer  personnel  from  each  side  participating.  This  could  go 
all  the  way  done  to  a  one  on  one  situation  where  the 
contracting  officer  debriefs  a  contracting  official  from  the 
unsuccessful  firm. 

The  debriefing  may  be  conducted  by  the  contracting 
official  alone  or  he  may  be  joined  by  other  personnel, 
especially  for  large  procurements.  For  these  large 
procurement  debrief ings  the  contracting  officer  is  responsible 
for  the  set  up  and  organization  of  the  debriefing.  He  or  the 
source  selection  evaluation  board  chairman  is  usually  the 
official  that  conducts  the  introduction  and  general  portions 
of  the  debriefing.  The  source  selection  evaluation  board 
factor  chairmen  or  representatives  will  conduct  a  debrief  on 
their  portion  of  the  source  selection  and  answer  any  related 
questions. 
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The  information  that  is  given  out  in  the  debriefing  is 
the  most  important  aspect  of  the  debriefing.  Obtaining  this 
information  is,  of  course,  why  the  unsuccessful  offerors  ask 
for  the  debriefing.  They  generally  look  for  as  much 
information  as  they  can  get  to  understand  why  they  didn't  win 
the  contract  award.  Most  AMC  officials  say  they  debrief  in 
accordance  with  the  RFP,  although  the  high  percentage  that  say 
they  do  is  disputed  by  the  unsuccessful  offerors.  The 
debriefing  will  address  the  technical,  management  and  cost 
merits  of  the  proposal  and  in  most  cases  identify  weaknesses 
in  the  offeror's  proposal.  Debriefing  officials  may  sometimes 
discuss  the  winner's  proposal,  but  only  in  very  general  terms. 
Details  of  the  winning  proposal  are  not  disclosed  nor  are 
comparisons  made  between  the  offeror's  proposal  and  other 
offerors'  proposals,  to  include  the  winner's  proposal.  In 
addition,  proprietary  business  data  from  other  offerors  is 
never  discussed. 

The  unsuccessful  offeror  is  permitted  to  ask  questions 
at  the  debriefing.  Questions  pertaining  to  the  offeror's 
proposal  are  answered,  although  detailed  or  difficult 
questions  may  result  in  an  answer  being  provided  sometime 
after  the  conclusion  of  the  debriefing.  Questions  pertaining 
to  another  offeror's  proposal  are  normally  not  answered.  They 
may  be  answered  if  they  are  of  only  a  very  general  nature, 
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result  in  no  point  by  point  comparisons  and  don't  involve 
proprietary  business  information. 

2.   Major  Items  Identified  To  Improve  Debriefing  Process 

The  research  shows  there  are  many  areas  of  the 
debriefing  process  that  can  be  improved.  Many  areas  for 
improvement  have  been  mentioned  several  times  by  the 
respondents  throughout  the  research  effort.  These  areas  are 
summarized  here. 

•  Debrief ings  should  be  conducted  in  a  timely  manner. 

•  No  limits  should  be  placed  on  who  the  unsuccessful 
offerors  send  to  debrief ings. 

•  Contracting  officers  should  debrief  in  an  outline  similar 
to  the  RFP. 

•  Debriefing  packets  should  be  provided  to  unsuccessful 
offerors  and  the  packets  should  be  given  to  them  before 
the  debriefing  if  possible. 

•  Unsuccessful  offerors  should  submit  questions  before  the 
debriefing  is  conducted. 

•  SSEB  members  and  other  evaluators  should  be  available  to 
participate  in  the  debriefing. 

•  The  strengths  as  well  as  the  weaknesses  of  the  offeror's 
proposal  should  be  covered  in  debrief ings. 

•  The  ratings  of  the  offeror's  and  winner's  proposal 
evaluations  should  be  disclosed. 

•  Narrative  comments  should  be  specific  and  be  given  for  all 
evaluated  areas . 

•  General  comments  on  the  winner's  proposal  evaluation 
should  be  discussed. 
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•  Some  comparisons  should  be  made  between  the  winner's  and 
unsuccessful  offeror's  proposals. 

•  The  basis  for  the  award  decision  should  be  revealed. 

•  Some  regulatory  restrictions  should  be  lifted. 

•  Specific  guidance  should  be  developed  which  covers  what 
should  and  shouldn't  be  discussed  in  debrief ings. 

•  Contracting  officers  should  release  the  source  selection 
decision  document  to  unsuccessful  offerors. 

•  Contracting   officers   should   offer   to   debrief   all 
unsuccessful  offerors. 

•  Contracting   officers   should   debrief   all   evaluated 
areas . 

•  Debrief ings   should  be  done  down  to  the   sub- factor 
level. 

•  Contracting   officers   should   discuss   the   evaluation 
rankings  for  the  unsuccessful  offeror  and  winner. 

E.   SUMMARY 

This  chapter  presented  and  analyzed  the  data  obtained  from 
the  surveys  and  interviews  conducted.  The  research  showed 
there  is  no  standard  debriefing  process  but  debriefings  are 
dependent  on  several  factors.  The  research  also  showed  that 
there  are  many  ways  the  debriefing  process  can  be  improved. 
Chapter  V  will  present  conclusions,  make  recommendations  to 
improve  the  debriefing  process,  answer  the  research  questions 
and  make  recommendations  for  further  research. 
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V.   CONCLUSIONS  AND  RECOMMENDATIONS 

A.  GENERAL 

There  are  several  conclusions  that  can  be  made  on  the 
current  debriefing  process  based  on  the  discussion  and 
analysis  of  the  research  done.  Many  recommendations  can  be 
made  to  improve  the  current  debriefing  process.  This  chapter 
will  address  the  conclusions  and  recommendations  on  the 
debriefing  process,  answer  the  research  questions  and  point 
out  areas  for  further  research. 

B.  CONCLUSIONS 

There  are  three  conclusions  that  can  be  made  based  on  the 
research  done. 

1.   Debrief ings  are  not  standard 

The  survey  data  shows  that  debrief ings  within  AMC, 
although  somewhat  similar,  are  not  standard.  Rather 
debrief ings  are  directly  influenced  by  a  variety  of  factors. 
The  main  factors  include  the  command  and  contracting  officer 
conducting  the  debriefing,  and  the  size  and  type  of 
procurement . 
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2 .  Current  debriefing  process  is  working 

The  current  debriefing  process  is  working.  The 
research  shows  that  respondents  get  some  value  out  of 
conducting  and  attending  debrief ings.  However,  the 
respondents  feel  that  the  current  debriefing  process  is  not 
working  as  well  as  it  could  and  improvements  could  be  made  to 
the  debriefing  process. 

3 .  Improvements  can  be  made  to  the  current  process 

The  research  shows  that  there  are  many  changes  that 
can  be  made  to  the  current  debriefing  process  that  can  improve 
debrief  ings.  Changes  can  be  made  by  both  unsuccessful 
offerors  and  contracting  officers  to  improve  the  process. 
These  changes  can  be  made  to  the  set  up  and  format  of  the 
debrief ings  and  also  the  debriefing  content. 

C .   RECOMMENDAT I ONS 

There  are  13  recommendations  that  can  be  made  based  on  the 
research  done. 

1.   Offer  debrief ings  to  all  unsuccessful  offerors 

Contracting  officers  should  offer  debriefings  to  all 
unsuccessful  offerors.  At  the  point  in  the  procurement 
process  where  the  debriefing  takes  place  the  contracting 
officer  is  providing  a  service  to  the  unsuccessful  offerors. 
To  provide  this  service  to  the  maximum  number  of  contractors 
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it  would  be  best  to  offer  debriefings  to  all  unsuccessful 
offerors.  The  regulations  do  not  require  AMC  to  offer 
debriefings  to  anyone.  They  only  state  that  debriefings  must 
be  given  to  those  offerors  that  request  a  debriefing  in 
writing.  However  offering  debriefings  to  all  unsuccessful 
offerors  will  ensure  that  contractors  unaware  of  their  chance 
to  be  debriefed  or  contractors  who  are  unsure  if  they  can  get 
a  debriefing  in  certain  procurements,  know  they  may  be 
debriefed.  Additionally,  offering  to  debrief  unsuccessful 
offerors  shows  courtesy  for  the  efforts  of  all  contractors 
attempting  to  win  the  contract  award.  Offering  debriefings 
also  shows  that  the  contracting  command  has  nothing  to  hide 
and  that  the  source  selection  was  done  rationally  and  in 
accordance  with  the  RFP. 

2 .   Debriefings  should  be  timely 

Debriefings  should  be  done  in  a  timely  manner.  The 
results  of  the  research  show  that  50%  of  all  respondents 
believe  debriefings  should  be  done  within  10  days  after 
contract  award.  Therefore  a  timely  debriefing  can  be 
considered  one  that  is  done  within  10  days  after  contract 
award.  Timely  debriefings  offer  several  benefits.  First 
there  is  a  better  chance  that  SSEB  members  and  other 
evaluators  who  can  enhance  the  quality  of  the  debriefing  will 
be  able  to  participate  in  the  debriefing.  The  longer  the  time 
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period  between  the  contract  award  and  the  debriefing  the  more 
likely  it  is  that  these  personnel  will  have  gone  on  to  other 
business  and  thus  not  be  available  for  the  debriefing. 
Additionally,  contractor  personnel  can  get  on  with  other 
business  quicker  when  the  debriefing  is  done  in  a  timely 
manner.  Debrief ings  that  are  done  quickly  can  also  boost  the 
confidence  that  unsuccessful  offerors  have  in  the  source 
selection  process.  By  providing  timely  debrief ings  AMC 
officials  show  that  they  are  organized,  professional,  and  can 
debrief  from  the  source  selection  evaluation  documentation 
without  needing  extra  time  to  prepare  additional  data  to 
justify  the  contract  award  or  proposal  evaluation.  Conducting 
debriefings  as  soon  as  possible  and  possibly  even  pre- 
scheduling  them  indicates  that  AMC  officials  believe 
debriefings  are  an  important  part  of  the  source  selection 
process  and  not  just  a  necessary  evil.  Timely  debriefings  may 
also  reduce  the  number  of  protests  that  are  filed.  A 
competent  debriefing  done  within  the  10  day  protest  window 
after  contract  award  should  show  the  contractor  that  his 
proposal  received  a  fair  evaluation  and  that  the  source 
selection  decision  was  proper.  A  debriefing  that  is  not  done 
until  after  this  window  closes  may  prompt  a  protest  before  the 
protest  window  closes  if  the  contractor  has  any  doubts  about 
the  fairness  of  the  selection  decision.   A  source  selection 
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that  is  protested  is  harder  to  debrief  and  provides  less 
information  than  one  that  is  not  protested  as  the  personnel 
doing  the  debriefing  would  be  much  more  conservative  with 
regard  to  what  information  they  release. 

3.   Place  no  limits  on  who  may  attend  debrief ings 

Contracting  commands  should  not  place  any  limits  on 
who  the  unsuccessful  offerors  may  send  to  debrief ings.  The 
unsuccessful  offeror  is  in  the  best  position  to  determine  who 
he  needs  to  send  to  the  debriefing  to  get  the  information  he 
desires.  Placing  limits  on  the  number  of  personnel  or  type  of 
employees  he  may  send  can  have  an  adverse  impact  on  what  the 
contractor  extracts  from  the  debriefing.  Placing  limits  on 
who  can  attend  can  also  invite  unwarranted  conclusions  from 
the  unsuccessful  offeror.  He  may  feel  that  the  contracting 
command  is  trying  to  hide  something  by  limiting  who  may  attend 
or  he  may  feel  that  he  would  have  gotten  more  from  the 
debriefing  if  he  could  have  sent  all  the  personnel  he  wanted 
to  send.  AMC  officials  need  to  keep  the  debriefing  process  as 
credible  as  possible  and  allowing  contractors  to  send  who  they 
want  to  send  to  the  debriefing  helps  in  this  regard.  Thus, 
AMC  personnel  should  fight  to  overcome  any  obstacles  that  may 
place  limitations  on  who  unsuccessful  offerors  may  send  to 
debrief ings . 
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4.  Give  contractors  debriefing  packets 

Contracting  officers  should  give  unsuccessful  offerors 
debriefing  packets  and  provide  them  before  the  debriefing  if 
possible.  Debriefing  packets  would  improve  the  debriefing  in 
several  ways.  First,  contractors  could  shift  their  attention 
from  the  distracting  task  of  copying  charts  and  slides  and 
taking  notes  to  listening  to  the  content  of  the  debriefing  and 
taking  a  few  notes.  Additionally,  the  unsuccessful  offeror's 
contracting  officials  will  have  an  official  record  of  the 
debriefing  from  which  they  can,  in  turn,  debrief  their 
superiors.  They  can  also  use  it  to  refer  to  later  if 
questions  arise  concerning  the  debriefing.  Next,  if  the 
debriefing  packet  is  given  to  the  contractor  before  the 
debriefing  it  would  allow  the  contractor  to  prepare  for  the 
debriefing.  They  could  study  the  advance  material  and  tailor 
their  interest  and  questions  to  areas  they  don't  understand  or 
agree  with.  Contractors  could  also  tailor  who  they  send  to 
the  debriefing  based  on  the  advance  material  they  get. 

5.  Evaluators  should  participate  in  debrief ings 

The  SSEB  members  and  any  other  evaluators  should 
participate  in  the  debrief ings.  They  have  the  expertise  and 
intimate  knowledge  of  the  proposal  evaluation  to  best  debrief 
or  answer  questions  in  their  respective  evaluation  areas. 
Debriefings  can  be  much  more  informative  when  evaluators  are 
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available  to  participate  in  the  debriefing.  Since  contracting 
officers  do  not  have  the  same  level  of  expertise  or  knowledge 
about  the  proposal  evaluation,  debrief ings  done  solely  by  the 
contracting  officer  tend  to  be  dry  and  uninformative  and  in 
many  cases  scripted.  Additionally,  the  contracting  officer 
would  be  hard  pressed  to  give  any  data  other  than  what  is  in 
the  script  as  he  would  not  have  the  capability  to  provide  any 
more  information.  Therefore  in  an  attempt  to  give  as  much 
information  as  possible  to  the  unsuccessful  offeror  it  is  best 
to  have  the  evaluation  team  available  to  participate  in  the 
debriefing. 

6.  Submit  questions  before  debrief ings 
Unsuccessful  offerors  should  submit  questions  before 

the  debrief ings  are  conducted.  This  will  allow  contracting 
officers  to  properly  research  these  questions,  prepare 
satisfactory  answers  and  present  these  answers  in  the 
debriefing.  This  will  reduce  the  problem  of  asking  a  question 
during  the  debriefing  and  not  getting  an  adequate  answer 
because  there  was  not  enough  time  to  gather  the  necessary 
information  and  prepare  the  answer. 

7.  Brief  in  an  outline  similar  to  RFP 

The  debriefing  should  be  done  in  an  outline  similar  to 
the  RFP.  Interview  results  show  that  a  debriefing  done  in  an 
outline  similar  to  the  RFP  is  easier  to  follow.    The 
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contractor  can  go  right  down  the  RFP  and  his  proposal  as  it  is 
debriefed  and  follow  the  debriefing  rather  than  flipping 
pages  trying  to  stay  with  a  debriefing  that  skips  around. 
Additionally,  according  to  one  contracting  officer,  using  an 
outline  similar  to  the  RFP  makes  it  easier  for  the  contracting 
officer  to  put  together  and  format  the  debriefing  while 
ensuring  all  relevant  areas  are  covered. 

8.  Cover  strengths  and  weaknesses 

Debriefings  should  cover  the  strengths  as  well  as  the 
weaknesses  of  the  offeror's  proposal.  Not  only  is  it 
important  to  discuss  where  the  unsuccessful  offeror's 
weaknesses  are  in  their  proposal  but  it  is  also  important  to 
discuss  their  strengths.  Revealing  the  offeror's  strengths 
will  let  the  contractor  know  where  the  contracting  command 
believes  the  offeror  is  particularly  strong.  This  will  allow 
the  contractor  to  not  waste  resources  by  making  unnecessary 
modifications  to  his  strengths  and  to  capitalize  on  these 
strengths  in  future  proposals. 

9.  Give  the  offeror  his  ratings 

Contracting  officers  should  give  the  unsuccessful 
offerors  their  ratings  for  each  of  the  evaluated  areas.  The 
ratings,  combined  with  good  narrative  comments,  will  give 
offerors  a  better  understanding  of  how  their  proposal  stood  up 
to  the  evaluation  compared  to  a  debriefing  where  just  comments 
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are  given  out.  As  an  example  an  evaluated  area  that  has  three 
positive  comments  and  one  negative  comment  with  a  rating  of 
outstanding  would  indicate  that  the  offeror  was  strong  in  that 
area.  The  same  comments  with  no  rating  would  leave  the 
offeror  with  a  question  of  how  strong  his  proposal  was  in  that 
area. 

10.  Debriefing  should  cover  all  evaluated  areas 

The  debriefing  should  cover  all  evaluated  areas. 
Unsuccessful  offerors  should  receive  a  better  briefing  if  all 
evaluated  areas  are  covered  in  the  debriefing.  The 
contracting  officer  and  others  who  debrief  are  familiar  with 
all  evaluated  areas  and  have  the  evaluation  comments  at  their 
disposal.  They  should  pass  on  their  evaluation  remarks  for 
all  areas.  Telling  an  unsuccessful  offeror  that  he  was 
satisfactory  in  an  area  and  then  following  up  with  any 
comments  noted  from  the  evaluation  is  more  informative  than 
bypassing  satisfactory  areas  and  leaving  it  up  to  the 
contractor  to  guess  how  he  did  in  the  bypassed  areas. 

11.  Debrief  down  to  sub -factor  level 

Not  only  should  personnel  conducting  the  debriefing 
brief  each  evaluated  area  but  they  should  brief  down  to  the 
sub- factor  level.  In  order  to  be  specific  enough  for  the 
remarks  to  be  helpful  and  informative  the  debriefing  should  go 
to  this  level.   A  common  complaint  from  the  unsuccessful 
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offerors  is  that  debrief ings  are  too  generic  in  nature. 
Briefing  down  to  the  sub- factor  level  with  good  narrative 
comments  will  help  alleviate  this  problem. 

12.   Use  the  latitude  authorized  to  deviate  from  AFARS 
when  in  best  interests  of  parties  involved 

Contracting  officers  should  use  the  latitude  they  have 
to  deviate  from  the  strict  guidance  of  the  AFARS  when  it  is  in 
the  best  interests  of  AMC  to  do  so.  A  conservative 
interpretation  of  the  AFARS  is  that  no  information  should  be 
given  out  on  any  competitor's  proposal  or  evaluation  of  their 
proposals  and  that  no  comparisons  will  be  made  between 
proposals .  The  research  has  shown  that  the  authors  of  the 
AFARS  intended  to  allow  contracting  commands  to  deviate  from 
the  guidance  of  the  AFARS  when  it  made  sense  to  do  so. 
Additionally,  several  commands  routinely  deviate  from  this 
guidance  to  get  as  much  information  to  the  unsuccessful 
offerors  as  they  can.  This  includes  giving  out  general 
details  of  the  winner's  proposal  and  the  evaluation  done  on  it 
and  making  some  general  comparisons  between  the  winner's  and 
unsuccessful  offeror's  proposals.  This  additional  information 
will  help  the  unsuccessful  offerors  understand  how  their 
proposal  stood  up  in  a  general  nature  to  the  winner's 
proposal . 
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13 .   Release  the  SSA  decision  document 

Contracting  officers  should  release  the  SSA  decision 
document  to  unsuccessful  offerors  during  the  debriefing. 
Research  has  shown  that  releasing  this  document  to 
unsuccessful  offerors  provides  contractors  with  information 
they  clearly  want  and  is  useful  to  them.  It  has  also  shown  to 
add  credibility  to  the  source  selection  process  in  the  eyes  of 
the  contractors.  The  document  must  be  scrubbed  to  ensure  any 
confidential  business  data  is  deleted  and  that  any  items 
expressly  prohibited  by  the  FAR  are  not  included  in  the 
document  that  is  released.  This  document  should  show  the 
basis  for  the  award  decision.  Items  that  could  be  released  in 
this  document  include  the  ratings  for  each  area  for  the 
winner's  and  offeror's  proposals,  some  general  comparisons  and 
why  the  decision  was  made.  Comments  such  as  the  following 
would  be  useful:  Company  A' s  proposal  was  good  but  not  as 
advantageous  to  the  Army  as  company  B's  proposal.  Company  B 
was  slightly  better  than  company  A  in  the  technical  area. 
Company  A  was  better  in  the  area  of  past  performance.  Company 
B  was  slightly  better  in  the  management  area  than  company  A. 
Company  B's  advantages  in  the  technical  area  and  management 
more  than  offset  company  A' s  advantage  in  past  performance 
therefore  company  B  was  chosen  for  the  contract  award. 
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D.   ANSWERS  TO  RESEARCH  QUESTIONS 

1.  Primary.  What  are  the  key  problems  and  issues 
associated  with  AMC's  process  of  debriefing  unsuccessful 
offerors  and  what  can  be  done  to  improve  the  process? 

Key  problems  with  the  debriefing  process  include 
debrief ings  not  being  done  in  a  timely  manner,  key  personnel 
from  AMC  not  attending  the  debriefing,  limits  being  placed  on 
who  may  attend  debrief ings  from  unsuccessful  firms,  and 
limited  information  being  given  out  in  the  debrief ings. 

The  process  can  be  improved  by  offering  debrief  ings  to  all 
unsuccessful  offerors,  conducting  debrief ings  in  a  timely 
manner,  not  limiting  who  may  attend  debrief ings,  providing 
unsuccessful  offerors  with  debriefing  packets,  and  providing 
more  information  to  the  unsuccessful  offerors. 

2.  Subsidiary. 

a)  What  is  the  current  debriefing  process  used  by 
AMC? 

There  is  no  standard  debriefing  process  used  by 
AMC.  Rather  AMC  MSC's  and  contracting  officers  tailor 
debrief ings  based  on  the  guidance  in  the  FAR,  AFARS,  internal 
operating  instructions  and  situation. 

There  are  some  generic  procedures  for  AMC 
debrief ings  however.  Debrief ings  must  be  conducted  for  all 
unsuccessful  offerors  that  ask  for  a  debriefing  in  writing. 
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The  debriefings  are  held  at  the  contracting  command  and 
several  people  representing  the  contractor  and  contracting 
command  normally  attend  the  debriefing.  The  contracting 
command  will  provide  the  unsuccessful  offeror  with  information 
that  will  help  the  offeror  determine  how  their  proposal  stood 
up  to  the  evaluation  criteria  and  why  they  did  not  win  the 
contract  award. 

b)  What  are  the  problems  associated  with  AMC's 
current  debriefing  process  from  AMC's  perspective? 

Several  problems  were  identified  with  the 
debriefing  process  according  to  AMC  contracting  officers. 
First,  sometimes  the  key  players  in  the  source  selection 
process  are  not  available  to  participate  in  the  debriefing. 
Next,  limited  information  is  put  out  in  the  debriefing.  Many 
debriefings  do  not  cover  all  evaluated  areas,  give  narrative 
comments  for  all  areas,  reveal  ratings  for  the  offeror's 
evaluated  areas,  make  any  comments  about  the  winner's 
proposal,  and  make  any  comparisons  between  the  winner's  and 
offeror's  proposals.  Additionally  debriefings  are,  at  times, 
not  timely. 

c)  What  are  the  problems  associated  with  AMC's 
current  debriefing  process  from  the  unsuccessful  offeror's 
perspective? 
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There  are  many  problems  with  the  debriefing 
process  according  to  unsuccessful  offerors.  Debrief ings  are 
not  done  in  a  timely  manner  and  do  not  provide  enough  details. 
Information  is  limited  due  to  restrictions  imposed  by  the 
regulations.  Little  or  no  information  is  given  out  on  the 
winner's  proposal.  Sometimes  the  main  players  in  the  source 
selection  process  are  not  in  attendance  at  the  debrief ings. 

d)  How  can  the  debriefing  process  be  modified  to  help 
AMC  and  the  unsuccessful  offerors? 

There  are  several  ways  to  modify  the  debriefing 
process  to  help  the  parties  involved.  Debrief ings  should  be 
offered  to  all  unsuccessful  offerors.  Debrief ings  should  be 
done  in  a  timely  manner  and  there  should  be  no  limits  placed 
on  who  may  attend  the  debrief ings.  Contracting  commands 
should  give  unsuccessful  offerors  debriefing  packets  and 
evaluators  should  participate  in  the  debrief ings. 
Unsuccessful  offerors  should  submit  questions  before  the 
debriefing.  The  debriefing  should  be  conducted  in  an  outline 
similar  to  the  RFP.  More  information  should  be  given  out  in 
the  debrief ings.  This  includes  debriefing  the  offeror's 
strengths  and  weaknesses,  giving  the  offeror  his  ratings, 
covering  all  evaluated  areas  and  debriefing  down  to  the  sub- 
factor  level  and  releasing  the  SSA  decision  document. 
Contracting  officers  should  also  use  the  latitude  they  have  to 
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deviate  from  the  strict  guidance  of  the  AFARS  when  it  is  in 
the  best  interests  of  the  parties  involved. 

E.   AREAS  FOR  FURTHER  RESEARCH 

An  area  for  further  research  can  be  to  look  at  the 
debriefing  process  by  contractor  commodity  type,  ie.  hardware, 
software,  services,  and  compare  these  commodity  types  to  see 
if  the  contractors  in  each  type  face  the  same  problems  with 
debriefings  or  if  debriefings  in  different  commodities  have 
different  limitations.  Another  approach  to  take  with  further 
resea  i  could  be  to  research  debriefings  by  large  and  small 
businesses  to  see  if  the  two  groups  face  the  same  or  different 
problems  with  debriefings. 
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APPENDIX 
SURVEY  FOR  UNSUCCESSFUL  OFFERORS  CONCERNING  ARMY  DEBRIEFING 


1.   When  not  awarded  a  contract  I  request  a  debriefing 

a.  more  than  2/3 's  of  the  time. 

b.  between  1/3  and  2/3 's  of  the  time. 

c.  less  than  1/3  of  the  time. 
Comments: 


2.   The  Army  gives  a  debrief  every  time  I  request  one 

a.  yes 

b.  no 
Comments: 


3.   The  following  personnel  (by  job  title)  from  my  company 
attend  the  debriefing. 

Comments : 


4.   The  Army  limits  the  number  of  employees  that  may  attend 
the  debriefing  from  my  company. 

a.  always 

b.  sometimes 

c .  never 
Comments : 


5.   The  Army  limits  the  employees,  by  job  position,  that  may 
attend  the  debriefing  from  my  company. 

a.  always 

b.  sometimes 

c .  never 
Comments: 
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6.  Debrief  ings  are  normally  held  within calendar  days  of 

my  request. 

a.  0-10 

b.  11-20 

c.  more  than  20 
Comments: 


7.  Debrief ings  should  be  conducted  within  calendar 

days  after  contract  award. 
Comments : 


8.   normally  debriefs  me. 

a.  the  contracting  officer 

b.  source  selection  evaluation  board  chairman 

c.  a  combination  of  both 

d.  other  

Comments: 


9.   Debrief ings  are  normally  held 

a.  at  my  location. 

b.  at  the  Army's  location. 

c.  other  location. 

d.  by  telephone 
Comments : 


10.   The  Army  

a.  doesn't  allow  questions. 

b.  allows  and  answers  all  questions. 

c.  allows  and  answers  some  questions. 
Comments : 


11.   If  the  Army  doesn't  answer  some  of  my  questions  it  is 
because 

a.  they   can't   answer   them  due   to   regulations   or 
confidential/proprietary  business  information. 

b.  they  won't  answer  them  due  to  their  concern  for  a 
protest. 

c .  other 

d.  not  applicable,  they  answer  all  my  questions. 
Comments : 
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12.   Debrief ings  normally  consist  of 
a.  limited  information,  because 


b.  as  much  information  as  the  Army  can  give  in  accordance 
with  the  regulations  and  confidential/proprietary  business 
information 
Comments :  


13.   The  Army  would  provide  more  information  if  the  possible 
threat  of  a  protest  did  not  exist. 

a.  always 

b.  sometimes 

c.  never 

d .  don ' t  know 

Comments:  


14.  The  Army  debriefs  in  accordance  with  the  major  elements  of 
the  RFP. 

a.  always 

b.  sometimes 

c .  never 
Comments: 


15.   Debrief ings  clearly  identify  weaknesses  in  my  proposal 

a .  always 

b.  sometimes 

c .  never 
Comments: 


16.    The  Army  debriefs  me  on  the  merits  of  my 

proposal  (circle  all  that  apply) . 

a.  technical 

b .  management 

c.  cost 
Comments : 


17.   I  am  satisfied  with  the  debriefing  on  the  technical, 
management,  and  cost  elements  of  my  proposal. 

a .  always 

b.  sometimes 

c.  never 
Comments : 
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18.    Debrief ings  give  me  information  that  leads  to  more 
competitive  proposals  in  future  Government  competition. 

a .  always 

b.  sometimes 

c .  never 
Comments : 


19.  Upon  conclusion  of  a  debrief,  I  completely  understood  why 
my  company  did  not  win  the  contract  award, 
a.  always 


b.  sometimes 

c.  never,  why  not_ 
Comments : 


20.   The  debrief  ings  are  to  my  company, 

a.  valuable 

b.  somewhat  valuable 

c.  not  at  all  valuable 
Comments: 


21.   The  Army  offers  to  debrief  me  even  when  I  don't  ask  for 
a  debriefing. 

a .  always 

b.  sometimes 

c .  never 
Comments: 


22.    Debrief ings  should  address   (circle  all  that  apply) 

,  as  long  as  confidential  business  information  is 

not  disclosed. 

a.  the  basic  proposed  technical  solution  of  the  awardee 

b.  the  overall  evaluated  cost  of  the  awardee  and  debriefed 
offeror 

c.  cost  or  price  associated  with  the  major  components  of 
the  awardee 's  proposal 

d.  overall  ranking  and  total  evaluation  scores  of  the 
awardee  and  debriefed  offeror 

e.  technical  point  scores  of  the  awardee  and  debriefed 
offeror 

f.  none  of  the  above 

g .  other 

Comments: 
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23.  The  sooner  I'm  debriefed  the  less  likely  I  am  to  protest 

a .  always 

b.  sometimes 

c .  never 

Comments :  


24.   List  3  strengths  of  the  Army  debriefing  process 


25.   List  3  weaknesses  of  the  Army  debriefing  process. 


26.   The  3  most  common  questions  I  ask  during  a  debriefing 


27.   The  Army  could  improve  the  debriefing  process  by 


28.    I/unsuccessful  offerors  could  improve  the  debriefing 
process  by 
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SURVEY  FOR  AMC  CONTRACTING  OFFICERS  CONCERNING  DEBRIEFINGS 


1.  How  many  unsuccessful  offerors  ask  for  a  debriefing? 

a.  less  than  1/3 

b.  between  1/3  and  2/3 

c.  more  than  2/3 

Comments : 

2.  I  give  a  debriefing  every  time  one  is  requested. 

a.  yes 

b.  no 
Comments : 


3.     The   following   personnel   (by   job   title)   from  my 
organization  attend  the  debriefing  

Comments : 


4.  I  limit  the  number  of  employees  that  unsuccessful  offerors 
may  send  to  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 
Comments : 


5.    I  limit  employees,  by  job  title,  that  unsuccessful 
offerors  may  send  to  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 
Comments : 


6.   Debrief ings  are  held  within  days  of  debriefing 

request . 

a.  0-10 

b.  11-20 

c.  more  than  20 
Comments : 


7.   Debrief ings  should  be  conducted  within  calendar 

days  after  contract  award. 
Comments : 
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8.   Debrief ings  are  conducted  by  (Give  job  title) 


Comments : 


9 .   Debrief ings  are  normally  held  

a.  at  my  location 

b.  at  the  unsuccessful  offeror's  location 

c.  other  location 

d.  by  telephone 
Comments : 


10.   I 


a.  don't  allow  questions 

b.  allow  and  answer  all  questions 

c.  allow  and  answer  some  questions 
Comments: 


11.    If  I  don't  answer  some  of  the  unsuccessful  offerors 
questions  it  is  because 

a.  I  can't  answer  them  due  to  regulations  or 
confidential/proprietary  business  information 

b.  I  won't  answer  them  due  to  a  possible  protest 

c.  other 


d.  n/a,  I  answer  all  questions 
Comments : 


12.   Debrief ings  normally  consist  of 
a.  limited  information,  because 


b.  as  much  information  as  I  can  give  in  accordance  with 
the   regulations   and   confidential/proprietary   business 
information 
Comments: 


13.   I  would  provide  more  information  if  the  possible  threat 
of  a  protest  did  not  exist. 

a .  always 

b.  sometimes 

c .  never 
Comments: 
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14.   I  debrief  in  accordance  with  the  major  elements  of  the 
RFP. 

a .  always 

b.  sometimes 

c .  never 
Comments: 


15.     Debrief ings   clearly   identify   weaknesses   in   the 
unsuccessful  offeror's  proposal. 

a .  always 

b.  sometimes 

c .  never 
Comments : 


16.  I  debrief  unsuccessful  offerors  on  the  merits  of 

their  proposal  (circle  all  that  apply) . 

a.  technical 

b .  management 

c.  cost 
Comments : 


17.  Unsuccessful  offerors  are  satisfied  with  the  debriefing 
on  the  technical,  management,  and  cost  elements  of  their 
proposal . 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 
Comments : 


18.  Debrief ings  give  unsuccessful  offerors  information  that 
leads  to  more  competitive  proposals  for  future  Government 
competition. 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 
Comments : 
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19 .   Upon  conclusion  of  a  debrief  the  unsuccessful  offeror 
completely  understands  why  he  didn't  win  the  contract  award. 

a .  always 

b.  sometimes 

c .  never 

d .  don ' t  know 

Comments:  


20.   The  debrief ings  I  give  are  to  unsuccessful 

offerors . 

a.  valuable 

b.  somewhat  valuable 

c.  not  at  all  valuable 
Comments : 


21.    My  organization  offers  to  debrief  all  unsuccessful 
offerors,  even  those  that  don't  request  a  debriefing. 

a .  always 

b.  sometimes 

c .  never 
Comments: 


22.   Debrief ings  should  address  (circle  all  that  apply) 


,  as  long  as  confidential  business 
information  is  not  disclosed. 

a.  the  basic  proposed  technical  solution  of  the  awardee 

b.  the  overall  evaluated  cost  of  the  awardee  and  debriefed 
offeror 

c.  cost  or  price  associated  with  the  major  components  of 
the  awardee 's  proposal 

d.  overall  ranking  and  total  evaluation  scores  of  the 
awardee  and  debriefed  offeror 

e.  technical  point  scores  of  the  awardee  and  debriefed 
offeror 

f .  none  of  the  above 

g.  other  

Comments : 


23.  The  sooner  a  debriefing  is  conducted,  the  more  likely  it 
is  to  prevent  a  protest. 

a .  always 

b.  sometimes 

c .  never 
Comments: 
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24.    My  organization  has  internal  instructions  covering 
debriefing  procedures 

a.  yes 

b.  no 
Comments : 


25.   List  3  strengths  of  the  Army  debriefing  process 


26.   List  3  weaknesses  of  the  Army  debriefing  process 


27.   I/The  Army  could  improve  the  debriefing  process  by 


28.   Unsuccessful  offerors  could  improve  the  debriefing  by 
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